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ABSTRACT 


The Asymmetrical Software Kit (ASK) is a software package built for U.S Army 
Special Operations Command (USASOC). It is designed to greatly expand and digitize 
the Intelligence Preparation of the Battlefield (IPB) process for Special Forces units. The 
purpose of this Thesis is to thoroughly evaluate the Tactical user’s acceptance of this 
technological innovation. Technology Acceptance Model, which psychometrically 
measures users’ perceptions of ease-of-use and utility to predict their intention to use the 
software, was applied in this analysis. The test population for this user acceptance survey 
is the Tactical (Group and below) level user of the ASK. These are the Special Forces 
Intel Sergeants (18Fs) on the Special Forces A-Teams (ODAs), and the Military 
Intelligence personnel at the Battalion and Group S2 (Staff Intelligence) sections. 
Respondents completed an anonymous, online survey on their impressions of the ASK. 
The questions were focused on system usability and user acceptance in a military setting. 
Overall, the models used in this study showed an acceptable level of fit with the Tactical 
end-user’s usability and acceptance assessments and exhibited satisfactory explanatory 
power. Users showed marked trends in response to questions concerning training, 
command involvement, and system availability. Qualitative input included a number of 
responses about the idiosyncrasies of certain programs, and the lack of high speed 
computers to run complex GIS queries. The findings from this study should provide 
some valuable insights to Program Managers about systems evaluation, and clarify how 
USASOC can design full spectrum software fielding to foster technology acceptance and 


use at the Tactical level. 
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I. INTRODUCTION 


Ax BACKGROUND AND MOTIVATION 


The Asymmetrical Software Kit (ASK) was designed using commercial off-the- 
shelf (COTS) software and hardware to aid the collection, processing, and dissemination 
of Military Intelligence in Army Special Operations units. The changing nature of 
warfare necessitated a newer, more agile analytic system to face newer, more agile foes. 
The Global War on Terror (GWOT) has highlighted the necessity for replacing legacy 
Intel systems like the All Source Analysis System (ASAS) and its younger sibling, the 
ASAS-Light. These systems, designed to operate in the Corps and Division Analysis and 
Control Elements (ACE), were left over from the Cold War. According to Army FM 34- 
25-3, the ASAS “is a "linchpin" system in forming a seamless intelligence architecture 
between and across echelons. They form a system that supports commanders from 


tactical through strategic levels anywhere across the range of military operations.”! 
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Figure 1. Army Intel Architecture and ASAS 


! Army FM 34-25-3, “All-Source Analysis System And The Analysis And Control Element,” 03 Oct 
1995. 


Engineered to process information on Warsaw Pact threats and push finished 
intelligence from Corps and Division analytic elements down to Brigade and below, they 
were wholly unsuited to the nature of the asymmetric and complicated threat encountered 
by Special Operations Forces in the GWOT. Most Intelligence requirements and 
procedures for SOF and conventional forces are the same. Yet, given the uniqueness of 
SOF missions and roles, there are characteristics of intelligence support that require 
unique systems and approaches. According to Army FM 3-05.102, Army Special 
Operations Forces Intelligence these are— 

¢ Level of detail. The intricate planning and decentralized execution of SOF 
missions require intelligence support that is frequently more detailed or specific than that 
needed in conventional operations. ARSOF often require intelligence to avoid enemy 
forces, regardless of size or composition, as opposed to information that would allow 
conventional forces to directly engage the enemy. 

¢ Timeliness and accuracy of information. Timely and accurate intelligence and 
unprocessed combat information can be a significant factor in operational success. Given 
the high risk factor or criticality of many ARSOF missions, uncertainty about the threat 
or environment reduces the chances for success more so than in the case of large-scale 
conventional operations. Equal with the need for accurate information is the need for 
timely information. Often the success of an ARSOF mission hinges upon having a key bit 
of knowledge at a specific time or event. 

¢ Operational element and analyst interface. Given the sensitivity and critical 
need for very detailed information, especially in the context of DA operations, 
intelligence personnel often need to work directly with the forces that will execute a 
mission. Frequently both analysts and collectors will work hand in hand with operational 
planners and even with those executing the actual mission during planning, rehearsal, 
execution, and post-operational phases of a mission. Because ARSOF units often use 
forward staging bases, assigned intelligence personnel must be familiar with the conduct 


of split-based operations. This allows intelligence personnel to deploy forward 


intelligence “packages” and to maintain connectivity with the Intelligence and Electronic 


Warfare (IEW) system through communications with their higher HQ. 


For SOF in an asymmetric fight, the legacy systems like ASAS failed because 
they are not able to meet the unique Intelligence needs critical to SOF mission success. 
The Asymmetrical Software Kit (ASK) was specifically commissioned to confront this 
change in the Military Intelligence paradigm that Army Special Operators were 
confronting in the GWOT. 


To meet the needs of the Special Forces Intelligence Sergeant, or 18F on the 
Special Forced Operational Detachment Alpha (ODA), the ASK _ was originally 
developed by cobbling together 3 COTS software programs - Analyst’s Notebook 
(ANB), Orion Magic (OM), and ArcGIS. These SOF peculiar Intelligence needs were 
calculated as being able to mine data (OM), analyze networks (ANB), and analyze and 
display data geospatially. In 2002, nothing existed in the military inventory. The Special 
Forces Intelligence Sergeant Course (SFISC) committee at the John F. Kennedy Special 
Warfare Center and School (SWCS) built their own package and insinuated it into the 
curriculum of the SFISC. The initial successes with the system and the analysis the 18F’s 
performed as part of their capstone exercise in the course began to be showcased to 
visiting dignitaries and General Officers. The software package quickly became fast 
tracked for acquisition and purchase by United States Army Special Operations 
Command (USASOC). In 2004, fielding to individual Special Forces, Civil Affairs, and 
other SOF units began.? 


Six years into the GWOT, and three years since the ASK’s initial fielding, 
evidence from the battlefield suggests that ASK may not be all that it forecast to be. In 
light of this evidence and the resulting concerns, it 1s appropriate to evaluate the ASK 


software package by focusing on those US military personal who that have used in on the 


2 Army FM 3-05.102, “Army Special Operations Forces Intelligence,” July 2001, pp. 2-5. 


3 MSG Matt Ames and CW3 Hendry during 01 May 2007 interview with author at Ft. Bragg, NC. 
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battlefield. The most important question before us is simple. Is ASK really working for 
American soldiers, and can we measure and quantify this through perceived ease of use 


and perceived utility? 


To date, most Information Technology (IT) research has concentrated on user 
acceptance factors in a business environment. In a market economy, there are a myriad 
of factors that encourage groups/ users to accept innovations. In a government 
economy, those factors differ greatly. According to Chau and Hu* and Lin°, the 
proliferation of innovative technologies that support individuals performing highly 
specialized tasks (like Intelligence Analysis) in the government sector is one area that 


needs additional research into technology acceptance decision-making. 


Investigations of technology usability and acceptance among tactical level (the 
soldiers on the pointy end of the spear) users have received precious little attention in 
previous research. In response, this project aims to examine user acceptance and 
usability of the ASK, across tactical Army SOF units. This study describes the design 
and implementation of a user-centric evaluation targeting system usability and user 


acceptance. 
B PROBLEM STATEMENT 


The ASK, fielded at the cost of millions of dollars, was designed to seamlessly 
integrate intelligence up, down, and across the Tactical levels of command. Anecdotally, 
the system has had major problems with full adoption. Users routinely leave the software 
and hardware in the rear when they deploy, or, at best, only deploy and use certain pieces 
of the system in the capacity they were designed for, leaving many powerful tools 


unused.© Why? 


4 Chau, P.Y.K. and Hu, P.J.-H. “Examining a model of information technology acceptance by 
individual professionals: An exploratory study,” Journal of Management Information Systems, 18(4), 2002, 
pp. 191-229. 


5 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, Doctoral Dissertation, University of Arizona, 2004. 


6 These scenarios came to light during the authors experience as an Intelligence Officer at 3rd Special 
Forces Group from 2004-2006, and during numerous research interviews conducted at Ft. Bragg in May 
2007. 
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C RESEARCH QUESTIONS 


Evidence suggests the Asymmetrical Software Kit is so complex, and the train-up 
is so daunting, that (as predicted by the Technology Acceptance Model), tactical users 
perceived utility, acceptance, and use of the system is low. If this is demonstrably true, 
how can we improve the software product to increase perceived value and ease of use 


thereby leading to a high level of user acceptance and use? 
D INVESTIGATIVE QUESTIONS 


1. Is ASK being accepted and used by tactical level producers and consumers of 


intelligence in USASFC (as measured by their survey responses)? 


2. With regards to ease of use, what, according to the tactical level users, are the 
problems endemic with the ASK? Do they relate to ease of use, social issues, or system 


availability? 


3. How can we increase user acceptance and use of these types of software 

systems?? 

a. What is the impact of increased training on user acceptance and use (18F 
Program of Instruction versus New Equipment Training)? 

b. Are there other alternatives (COTS/GOTS software) that contain attributes 
that will increase perceived value and ease of use? What are those attributes 
independent of a particular alternative? 

c. Howcan we better plan for and manage technology acquisition to increase 


acceptance and use? 
E RESEARCH FOCUS 


There are many different types of collaborative intelligence software that is being 
used in both the private (financial analysis) and government (national intelligence 
agencies) sectors. The focus of this research will be a thorough analysis of the factors 


impacting user acceptance of a collaborative intelligence software suite currently fielded 


by Army Special Operations units at the Tactical level. The results will be the statistical 
output derived from a proven theoretical model (Technology Acceptance Model) used to 
predict future usage of new Information Technology (IT) and specific user-input for 


improvements on the current and future systems the service will field. 


F METHODOLOGY 


To answer the key research questions, a case study will be performed to identify 
specific factors that impact user acceptance and usage of the ASK. This case study will 
be based on a comprehensive literature review and interviews with tactical level users at 
Ft. Bragg, NC. These findings will be used to construct the survey instrument used 
during the case study. A thorough statistical analysis of the survey data and user 
suggestions will be used to make suggestions for improvements to the current system, 


and generate requirements for fielding of future systems. 


G SCOPE 


The scope of this research effort will be to explore the numerous factors 
impacting acceptance and use of the Asymmetrical Software Kit among the tactical level 
users. To accomplish this, the research will review existing literature and studies to 
identify factors affecting participation in other forms of collaborative intelligence 
software. The goal of this will be to identify the key elements of gaining user “buy in” 
and participation in collaborative intelligence programs. The results will potentially be 
used to help modify and manage the existing intelligence systems, and assist in the 


design, implementation, and fielding of future software packages. 
H LIMITATIONS 


The limitations of this research include the small sample population (+/-300) of 
ASK users at the tactical level in USASOC units. This is due to the nature of the case 
study itself. Outside factors have a limiting factor on respondents. USASOC units are 


currently heavily engaged in the GWOT, and are deployed across the globe, making 


contact with an already small population even more difficult. In addition, this project is 
only studying the Asymmetrical Software Kit at the tactical level. As a result of this, the 
results of this study may not be applicable beyond the USASOC community or to IT 


acceptance theories in general. 
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Hl. LITERATURE REVIEW 


A. OVERVIEW 


This thesis research attempts to identify the factors affecting user acceptance of 
the Asymmetrical Software Kit (ASK) based on the theories of the Technology 
Acceptance Model (TAM). The scope of this review represents the learned opinions of 
numerous experts and academics from different books and journals pertaining to 
technology acceptance and use of Information Technologies (IT). The information in this 
literature review will attempt to explain the components and function of the ASK as well 


as the pertinent factors that affect technology acceptance and adoption. 
B. THE ASYMMETRICAL SOFTWARE KIT 


The ASK consists of three COTS software components: ArcGIS, Analyst’s 
Notebook, and Orion Magic. Each component serves as an integral piece of the 
Intelligence analysis that tactical users do on a continuous basis. The amount of data the 
average 18F takes in has increased exponentially in the information age. These software 
tools allow the SF Intelligence Sergeant and the Military Intelligence analysts he interacts 
with at the Special Forces Battalion and Group levels to quickly store, process, analyze, 
and visualize unprecedented amounts of information. The three components each work 


in their own way to facilitate this process. 


Data collection and organization is the job of SRA Incorporated’s Orion Magic 
(OM) software. OM is a “single framework for data collection, mining, and distribution. 
OM is a suite of tools developed to support investigative processes, from data collection 
to statistical analyses, to results visualization.’ Users can logically organize incoming 
data in a hierarchical format of cabinets, outlines, and notecards to speed up subsequent 
data mining efforts. OM is also equipped with a powerful search engine to search the 


data analysts have organized. Their “Matrix Search” feature “is a series of vertical 


7 Orion Magic website, http://www.orionmagic.com/overview.htm, accessed 19 September 2007. 
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column cells and horizontal row cells, each containing one or more keywords, (or 
concepts) that can be applied to the web, documents or databases and return results for 
not only each cell but also the intersection of those cells.”® Searching and displaying 
intersecting data in a format like this allows analysts to quickly visualize a few, 


significant intersecting data points in a mass of information. 
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Figure 2. File Cabinets and Outlines in Orion Magic 


8 Orion Magic website, http://www.orionmagic.com/search.htm, accessed 19 September 2007. 


9 Image downloaded from Orion Magic website, http://www.orionmagic.com/collect.htm, on 19 
September 2007. 
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Figure 3. Matrix Search results in OM 


Geo-visualization and analysis of data is another critical task for the SF 
Intelligence Sergeant and the MI analysts that support him. This is accomplished using a 
Geospatial Information System (GIS). “A Geographic Information System (GIS) is a 
system for the management, analysis, and display of geographic information. Geographic 
information is represented by a series of geographic databases that model geography 
using simple, generic data structures.” !! GIS is also defined as “‘an integrated system of 
computer hardware, software, and trained personnel linking topographic, demographic, 


utility, facility, image and other resource data that is geographically referenced.” !2 


ArcGIS is a product of ESRI, a leading software company in the GIS business. In 
the ASK, ArcGIS includes: ArcReader, which allows the user to view and query maps 
created with the other Arc products; ArcView, which allows users to view spatial data, 


create maps, and perform basic spatial analysis; ArcEditor which includes all the 


10Image downloaded from Orion Magic website, http://www.orionmagic.com/search.htm, on 19 Sep 
2007. 


!1 ESRI white paper, “What is ArcGIS?,” 2004, accessed on 21 Aug 2007 at 
http://downloads.esri.com/support/documentation/ao_/698 What_is_ArcGIS.pdf . 


12 NASA — Langley GIS Research Team, definition found on NASA website 17 Sep 2007 at http://gis- 
www.larc.nasa.gov/qat/gisdefinition.html — 
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functionality of ArcView, includes more advanced tools for manipulation of shapefiles 


and geodatabases; or ArcInfo the most advanced version of ArcGIS, which includes 


added capabilities for data manipulation, editing, and analysis. To increase the 


functionality of ArcGIS, 


special Military 


“Extensions” 


have been created to 


accommodate military operational symbols, military grid referencing systems (MGRS) 


and digital terrain elevation data (DTED).!3 
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Data table in MS Excel format 


13 ESRI website, http://www.esri.com/software/arcgis/extensions/militaryanalyst/index.html. accessed 


209 September 2007. 
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Figure 5. Data graphically displayed and overlaid on mapping 


Analyst’s Notebook (ANB) is social network analysis software from the [2 
Corporation. It allows analysts to process, analyze, and graphically display immense 
amounts of data on networks. The Analyst's Notebook also enables analysts to prepare 
and share link-analysis charts. Figure 6 shows a sample link analysis chart (finished 
intelligence) from ANB. SOF Intelligence analysts use ANB to identify links between 
known or suspected terrorists, their activities, phone numbers, locations, and their 
associations with other persons, events, or groups. With ANB, data trends and linkages 


can be visually displayed, interpreted, and subsequently analyzed. 


14 Images downloaded from ESRI Website, 
http://www.esri.com/software/arcgis/about/desktop_gis.html on 18 Sep 2007. 
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Figure 6. Sample Link Analysis Chart 


C. TECHNOLOGY ACCEPTANCE THEORIES 


In the following sections, the preeminent theories on technology acceptance and 
adoption will be reviewed. Classical Diffusion of Innovation (DOI) theory and the 


leading intention-based theories that provide theoretical underpinnings of this study will 


be the main focus. 
D. DIFFUSION OF INNOVATION 


The model of “diffusion of innovation” was proposed by Everett Rogers in 1962. 


It was originally used by rural sociologists to study the diffusion of agricultural 


13 Image downloaded from I2 corporate website, 
http://www.i2inc.com/Products/Analysts_Notebook/, accessed 19 Sept. 2007. 
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technologies in social systems, but it has stood the test of time and been very successfully 


applied to information technology products. 16 


According to Rogers, innovation is “an idea, practice, or object perceived as new 
by an individual or other unit of adoption” !/ and diffusion is “the process by which an 
innovation is communicated through certain channels over time and among the members 
of a social system.” 18 Rogers used well-established theories in sociology, psychology, 


and communications to develop a clear and succinct approach to explaining the diffusion 


of innovations. 


The Innovation Adoption Curve 
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Figure 7. Innovation Adoption Curve 


After its conception, an innovation spreads slowly at first - usually through the 


campaigning of change agents - then picks up speed and visibility as more and more 


16 E. Mustonen-Ollila, & K. Lyytinen, (2003). Why Organizations Adopt Information System Process 
Innovations: A Longitudinal Study using Diffusion of Innovation Theory, Information Systems Journal, 
13(3), pp. 275-297. 


17 Everett M. Rogers, Diffusion of Innovation, New York, Free Press, 2003, p. 12. 
18 Everett M. Rogers, Diffusion of Innovation, New York: Free Press, 2003, p. 35. 


19 Image downloaded from Tulane University website, http://www.payson.tulane.edu/research/E- 
DiffInnova/diff-prob.html accessed 14 September, 2007. 
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people are exposed to and adopt the innovation. Eventually it reaches a saturation level, 
where nearly everyone who is going to adopt the innovation, minus the hardcore luddites, 


has done so. 


Individuals within a social system do not all adopt an innovation at the same time, 
but rather in an over-time sequence, so that individuals can be classified into adopter 
categories based on when they begin using an innovation. Socioeconomic and individual 
personality characteristics define the individual’s role in the diffusion process. These 
adopter roles are: innovators (2.5%), early adopters (13.5%), early majority (34%), late 
majority (34%), and laggards (16%). 


25% 16% 


Innovators = Early Early Late Lagqards 
Adopters Majority Majority 





Figure 8. Adopter Categorization on the Basis of Innovativeness 


After the forward-thinking change agents have adopted the innovation, they work 
to communicate it to others in the society by whatever means they believe appropriate. 
When the number of early adopters reaches a “critical mass” - between 5 and 15% - the 


process becomes self-sustaining, as more and more people talk about or demonstrate the 


20Everett M. Rogers, Diffusion of Innovation, New York, Free Press, 2003, pp. 280-281. 
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innovation to each other.?! Innovation diffusion theorists have identified five perceived 
attributes?? that explain why some innovations reach this tipping point and others never 
do. They are: 

Relative Advantage - Is the innovation better than the status quo? Will people 
perceive it as better? If not, the innovation will not spread quickly, if at all. 

Compatibility - How does the innovation fit with people's past experiences and 
present needs? If it doesn't fit both well, it won't spread well. Does it require a change in 
existing values? If members of the culture feel as though they have to become very 
different people to adopt the innovation, they will be more resistant to it. 

Complexity - How difficult is the innovation to understand and apply? The more 
difficult the innovation is to grasp, the slower the adoption process. 

Trialability - Can potential adopters "try out" the innovation first, or must they 
commit to it all at once? If the latter, people will be far more cautious about adopting it. 

Observability - How visible/tangible are the benefits of adoption? If people adopt 
it, can the difference be discerned by others? If not, the innovation will spread more 


slowly.23 
E. INTENTION-BASED ACCEPTANCE THEORY 


Technology Acceptance Modeling (TAM) builds on Rogers’ research on patterns 
of diffusion by providing a reliable method for predicting user behavior. Three models 
have evolved to successfully explain actual system use by measuring users’ perceptions 
towards the innovation. These are the Theory of Reasoned Action (TRA), Theory of 
Planned Behavior (TPB), and Technology Acceptance Model (TAM). 


According to Fishbein and Ajzen’s TRA, people’s beliefs influence their attitudes, 
which together with subjective norms shape their behavior intentions. This then guides 


or dictates their actual behavior. In numerous studies, intention and actual behavior have 


21 Everett M. Rogers, Diffusion of Innovation, New York, Free Press, 2003, p. 344. 


22 Perception of adopter is key in Technology Acceptance Model (TAM) discussed under intention- 
based theories. DOI is the antecedent to TAM. 


23 Everett M. Rogers, Diffusion of Innovation, New York, Free Press, 2003, p. 265-266. 
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been shown to be correlated highly. TRA has been applied scientifically to explain a 
wide variety of behaviors, including consumer purchasing2*, condom usage?°, and breast 
self-examination.2© TRA forms the basis for the Technology Acceptance Model 


ultimately selected for use in this study. 


Beliefs and Attitude 
Evaluations Toward 


Behavioral Behavior 
Intention 


Normative Beliefs & Subjective 
Motivation to Comply : Norm 





Figure 9. Theory of Reasoned Action?7 


Icek Ajzen’s Theory of Planned Behavior (TPB) posits that attitudes towards 
behavior and the subjective norm from TRA are insufficient to explain behavioral 
intentions. TBP introduces yet another factor, Perceived Behavioral Control (PBC) to the 
mix. PBC measures of the amount of control people perceive they have over their own 
behavior. As a general rule, the more favorable the attitude toward behavior and 
subjective norm, and the greater the perceived behavioral control, the stronger the 
person’s intention to perform the behavior in question should be. Finally, given a 
sufficient degree of actual control over the behavior, people are expected to carry out 


their intentions when the opportunity arises. 2° 


24B_H. Sheppard., J. Hartwick, J.& P. R.Warshaw, The theory of reasoned action: A meta-analysis of 
past research with recommendations for modifications and future research, Journal of Consumer Research, 
vol 15, 1988, p. 325. 


25 Y, Kashima, C Gallois, & M. McCamish, The theory of reasoned action and cooperative behaviour: 
it takes two to use a condom, British Journal of Social Psychology, 1993 Sep;32 ( Pt 3): pp. 227-39 . 


26 LM. Lierman, H. M. Young, D. Kasprzyk, & J. Q. Benoliel, Predicting breast self-examination 
using the theory of reasoned action, Nursing Research, 1990 Mar-Apr;39(2): pp. 97-101. 


27 Image downloaded from Your University website 
http://www.istheory.yorku.ca/theoryofreasonedaction.htm on 07 Dec 2007. 


28]. Ajzen, Residual Effects of Past on Later Behavior: Habituation and Reasoned Action 
Perspectives. Personality and Social Psychology Review, 6(2), 2002, pp. 107-122. 
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Figure 10. Theory of Planned Behavior, from Environmental Health Journal. 


Fred Davis proposed Technology Acceptance Model (TAM) in 1989 as a logical 
extension of TRA and TPB. TAM theorizes that the two main beliefs determining 
attitude and behavioral intention are perceived usefulness (PU) and perceived ease of use 
(PEOU). USE is defined as the degree to which a user believes that the system he is 
adopting will enhance his or her performance. PEOU refers to the degree to which a user 
believes that using the new system will be free from physical and mental effort.3° The 
goal of TAM is to predict and explain user acceptance of Information Technology (IT) 
across a wide range of technologies, organizational contexts, and user populations. 
Given a choice between two systems, users will generally rate the system they find easier 
to use as having a higher PEOU score. TAM has been used successfully in numerous 
studies. Davis, Bagozzi, and Warshaw were able to successfully predict usage of a word- 
processing program in a class of graduate students?!, and Dillon and Morris had similar 
results predicting usage of a web-browser among computer science students.72 The basic 


TAM model is depicted below. 


29 Image downloaded from Environmental Health Journal at 
http://www.ehjournal.net/content/4/1/23/figure/F1 on 28 Sep 2007. 


30Fred Davis, Perceived usefulness, perceived ease of use, and user acceptance of information 
technology. MIS Quarterly, 13(3), 1989, pp. 319-340. 


31 Fred Davis, R. P.Bagozzi, & P. R. Warshaw, User Acceptance of Computer Technology: A 
Comparison of Two Theoretical Models, Management Science, 35(8), 1989, pp. 982-1003. 


32 A. Dillon & M. Morris, User Acceptance of Information Technology: Theories and Models, Annual 
Review of Information Science and Technology, 31, 1996, pp. 3-32. 
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Figure 11. Technology Acceptance Model 


F. EVALUATION METHOD SELECTION 


After researching the three models presented (TRA, TPB, and TAM), it was 
determined that TAM would best be able to predict future usage of the ASK for three 
reasons: (1) TAM’s legitimacy is firmly grounded in the literature of psycho-social 
sciences, (2) TAM was specifically developed to explain and predict user acceptance of 
technological innovations like the ASK, and (3) TAM’s ability to predict system usage 


with a minimum amount of cost and difficulty are ideal for a study like this. 
G. THE RESEARCH MODEL 


To date, only Chienting Lin, Paul Hu, and Hsinchun Chen have looked at usage of 
Intelligence and Security Informatics (ISI) systems (specifically the COPLINK system 
used by the Tucson PD) based on the Technology Acceptance Model.7* The model for 
this study is drawn from that work, and is an attempt to further validate their findings in a 


similar (military versus police) environment. 


33M. Morris & A. Dillon, How User Perceptions Influence Software Use. IEEE Software, 14(4), 1997, 
pp. 58-65.: 


34 Paul Hu, Chienting Lin,.& H. Chen, User acceptance of Intelligence and Security Informatics 
technology: A study of COPLINK, Journal of the American Society for Information Science and 
Technology, 2005; 56(3) pp. 235-244. 
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Figure 12. Asymmetrical Software Kit User Acceptance Study Research Model 


Research suggests that a user’s decision to accept or refuse an ISI technology can 
be explained by factor pertaining to the technological, social, and organizational contexts. 
By examining these three factors, one can determine the roots of some of the barriers to 


adoption in a community of users. 


The core principles of the technological context are perceived usefulness (PU), 
perceived ease of use (PEOU). For this study, PEOU and PU both determine attitude, 
and PEOU has a direct and positive effect on PU. Efficiency gain is an extremely 
important component in the Special Operations community: SOF peculiar intelligence 
requirements are, as previously noted in Chapter 1, Level of Detail, Timeliness and 
accuracy of information, and Operational element and analyst interface. Intelligence 
analysts at the tactical level continuously work against the clock to process, analyze, and 
disseminate intelligence from an ever increasing torrent of raw information. 


Accordingly, the following hypotheses are tested in this study. 
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Perceived Usefulness 


H1: The level of efficiency gain of the ASK as perceived by the tactical users 


will positively affect their perceived usefulness of the ASK. 


H2: The level of job relevance of the ASK as perceived by the tactical users will 
positively affect their perceived usefulness of the ASK. 


H3: The level of product quality of the ASK as perceived by the tactical users will 
positively affect their perceived usefulness of the ASK. 


H4: The level of external data access of the ASK as perceived by the tactical users 


will positively affect their perceived usefulness of the ASK. 
Perceived Ease of Use 


H5: The level of external data connectivity of the ASK as perceived by the 


tactical users will positively affect their perceived ease of use of the ASK. 


H6: The level of functionality of the ASK as perceived by the tactical users will 


positively affect their perceived ease of use of the ASK. 


H7: The level of learnability of the ASK as perceived by the tactical users will 


positively affect their perceived ease of use of the ASK. 


Through system exposure in training and daily job usage, users in the research 
population have developed personal assessments of, and attitudinal beliefs about the 
ASK. According to TAM and TPB, individuals embracing a positive attitude towards an 
innovation are more likely to accept that technology than those displaying a negative 
attitude. While others have questioned the effectiveness of attitude in mediating PEOU 
and PUs impact on Behavioral Intent?°, like Hu, Lin, and Chen, this study has elected to 
retain attitude because of the considerable autonomy afforded Military Intelligence 


analysts and SF Intelligence Sergeants. 


35 Fred Davis, Richard Bagozzi, & P. Warshaw, User Acceptance of Computer Technology: A 
Comparison of Two Theoretical Models, Management Science, 35(8), 1989, pp. 982-1003. 
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In the Social context of this study, subjective norm refers to the user’s assessment 
or perception of his significant referents’ desires or opinions regarding his or her 
acceptance of the ASK. In the military, this is the user’s peer group and his immediate 
chain of command. In the military, as in the law-enforcement community Lin studied, 
soldiers form psychological attachments to their unit and to the social system within it.>° 
This attachment, in the military context, can be attributed to personal commitment to 
public service, the sense of community in military units, and the great amount to which 
soldiers depend on each other in an increasingly dangerous environment. In this study, 
we measured that by looking at Peer Influence (PEER). Given these factors, a military 
user 1s especially likely to consider the ASK useful and exhibit a strong intention to 
accept it when his fellow analysts and his leaders are in favor of adopting the innovation. 
Since there is a hierarchy in the profession, a military user is also likely to take into 
consideration the wants and needs of his supervisors. This is measured in the Chain of 


Command (COC) portion of the survey. 
Subjective Norm 


H8: The level of peer influence (PEER) as perceived by the tactical users will 


positively affect the intensity of their perceived subjective norm towards accepting the ASK. 


H9: The level of command (COC) involvement as perceived by the tactical users will 


positively affect the intensity of their perceived subjective norm towards accepting the ASK. 


System availability (AV), Training (TNG), and Support (SPT) are key 
components of the organizational context. System availability refers to the users 
perception of the availability of equipment necessary for accessing and using the ASK. 
Resource availability is a fundamental construct of perceived behavioral control, a 
component of Ajzen’s TPB. In Determinants of Perceived Use, Venkatesh specifically 


notes the effect of availability of resources and opportunities on user acceptance.3’ In 


36 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, University of Arizona, 2004, p.75. 


37 Vv. Venkatesh, Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, 
and Emotion into the Technology Acceptance Model, Information Systems Research, 11(4), p. 342-365.: 


Zz 


addition to the previous research, the subjects of system speed, technical support, and 
computer availability were prevalent themes in pre-study interviews conducted with users 
at Ft. Bragg. There is also a large difference in the training that SOF and MI MOS 
soldiers receive on the system. In light of that information, the study tested the following 


hypotheses. 


Behavioral Intent 

H10: The level of system availability as perceived by the tactical users will positively 
affect the intensity of their behavioral intention towards accepting the ASK. 

H11: The quality of training as perceived by the tactical users will positively affect 
the intensity of their behavioral intention towards accepting the ASK. 

H12: The level of support as perceived by the tactical users will positively affect the 


intensity of their behavioral intention towards accepting the ASK. 
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Hl. METHODOLOGY 


A. OVERVIEW 


This chapter describes the overall methodology of this research project. The 
design of the qualitative study will be covered in depth to illustrate the processes and 


procedures used to perform the case study in this research project. 
B. TAM IMPLEMENTATION 


The following steps were derived from a comprehensive reading of previous 
research: (1) identify an appropriate study group, (2) develop /design the survey 
instrument, (3) gather data, (4) analyze the data, and (5) draw the appropriate 


conclusions. The following is a discussion of how these steps were implemented. 
C. IDENTIFY STUDY GROUP PARTICIPANTS 


The population of the study consists entirely of Military Intelligence personnel, 
Special Forces Intelligence Sergeants, and Civil Affairs Soldiers assigned to United 
States Army Special Operations Command (USASOC). They represent several different 
Special Forces Groups, the JFK Special Warfare Center and School, the 96" Civil Affairs 
Brigade, and soldiers in various subordinate headquarters units. These soldiers all have 
experience with the Asymmetrical Software Kit, and use it, or components of it, 1n the 
course of their analyst duties. They represent a broad range of ages, experience, job 


training, and education. 
D. INSTRUMENT DEVELOPMENT 


Multiple techniques were used during the instrument development phase. 


Candidate items for the survey were identified through previous research. The TAM 
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studies conducted on COPLINK by Lin, Hu and Chen?° and previous research done by 
Lin>? provided a readily validated survey instrument. The survey questions were drawn 
from this work, and then modified to reflect a change in communities (from police to 
military) and lexicons. After these changes, two Soldiers from the Ft. Bragg study group, 
and one Special Forces officer at the Naval Postgraduate School voluntarily assessed the 
validity of the resulting items at face value. From their input, a few minor changes were 
made to tailor the survey to the community. All question items (refer to Appendix B, 
survey questionnaire) were measured using a seven-point Likert scale. The possible 


99 


response ranged from “strongly agree” to “strongly disagree.” Multiple questions were 
used to measure single variables to compensate for how subjects respond to questions 
with certain word structures. To reduce the incidence of monotonous responses on the 
survey, the sequence of questions was randomized and half of the questions were 


negated. 
E. DATA COLLECTION 


The study empirically tested the research model using data collected from a self- 
administered, online survey. The link to the website was distributed to over 350 
participants through their chains of command. All subjects participated in the study 
voluntarily. The website with the survey was kept open for eight weeks, from 13 
September 2007 to 12 November 2007 to allow as many subjects, given the current unit 
operational tempo and competing deployments, to reply as possible. Of the Soldiers that 
it was emailed to, 38 initiated the survey on the website. Of those 38 hits, 25 produced 


useable data. 


The survey was designed to collect both qualitative and quantitative data. The 


constructs were measured using a seven point Likert Scale, with one as “Strongly 


38 Paul Hu, Chienting Lin, & H. Chen, User acceptance of Intelligence and Security Informatics 
technology: A study of COPLINK, Journal of the American Society for Information Science and 
Technology, 2005; 56(3) p. 235-244. 


39 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, University of Arizona, 2004. 


26 


Disagree” and seven as “Strongly Agree.” See Table | below for a sample question from 
the survey with the accompanying seven point Likert Scale below it. Multiple questions 
were used to measure single variables, allowing for variation in how individuals respond 
to varying word structures. The process of determining which sets of questions exhibit 
the smallest amount of variance for a particular variable is called Reliability Analysis. 
Reliability is calculated using a statistical procedure to measure Cronbach’s Alpha.*9 
Cronbach Alpha ranges between 0.0 and 1.0, and value of greater than 0.7 is considered 


sufficient for social research. 


12. Using the ASK would improve my job performance 


Neither 
Slightly Agree Slightly 
Agree nor Disagree 


Strongly 


Agree with Bere 


with the 


Disagree Strongly 
with the | Disagree with 
statement | the statement 


the ea ae with the | Disagree | with the 
statement statement | withthe | statement 


Statement 





Table 1. | Sample Survey Question*! 


In his initial research on TAM, Fred Davis performed this measure on each of his 
four variables, Perceived Utility (PU), Perceived Ease of Use (PEOU), Attitude (ATT) 
towards using, and Behavioral Intent to use (BI), to ascertain their reliability. His four 
measures of PU achieved a reliability score of 0.92, and PEOU achieved 0.90. The ATT 
scale and BI scale obtained reliability scores of 0.82 and 0.92 respectively.42 The same 


analysis was repeated on the data gathered for this study to ensure reliability. 


40L,. Cronbach, Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 1951, pp. 
297-334. 


41 VV. Venkatesh and F. Davis, A Model of Antecedents of Perceived Ease of Use: Development and 
Test. Decision Sciences, 27(3) 1996, pp. 451-481. 


42 Vv. Venkatesh, and F. Davis, A Model of Antecedents of Perceived Ease of Use: Development and 
Test. Decision Sciences, 27(3) 1996, pp. 451-481. 
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In addition to the TAM constructs investigated by Davis, this study also 
investigates users perceptions of system Functionality (FUNC), available Tech Support 
(SPT), the importance of External Connectivity (EXCON), Efficiency Gain (EG) and the 
Training (TING) they have received on the system. These questions were modeled after 


the studies on user acceptance of COPLINK conducted by Lin.*? 


To assess the qualitative aspect of the survey, three open ended questions were 
asked at the end of the survey. Respondents were elicited for their input on what part of 
the system worked well, what needed fixing, and their suggestions for future 


improvements. A sample of their input is included in Chapter IV. 


43 Chienting Lin, Examining Technology Usability and User Acceptance In Digital Government: A 
Case Study In Law Enforcement, University of Arizona, 2004. 
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IV. RESULTS AND ANALYSIS 


A. OVERVIEW 


This chapter presents the results of the research study in four distinct sections: 
Instrument Validation, Respondent Demographics, Qualitative Feedback, and 
Quantitative Analysis. The survey instrument was validated using Reliability Analysis. 
The subsequent sections examine respondent demographics; summarize some user- 
generated qualitative feedback about the system, and statistically analyze the survey data 


to find correlations to the model. 
B. INSTRUMENT VALIDATION 


Most of the items in this survey were drawn from previously validated 
measurements, but this study re-examines the instrument’s convergent and discriminate 
validity. Using the responses from the survey, the instrument’s convergent validity was 
examined using the Cronbach’s alpha value for each of the specified constructs. Many 
social science professionals, as a rule of thumb, require a Cronbach's alpha reliability 
score of 0.70 or higher.44 Cronbach's alpha is viewed as a measure of how well the sum 
score of the selected items capture the expected score for the larger domain, even if that 
domain is heterogeneous. The results of the reliability analysis are displayed in Table 3 
below. The Cronbach’s Alpha values ranged between .073 and .095, exhibiting more 
than adequate reliability. 


To evaluate the instrument’s discriminate validity, principal component analysis 
was performed using a varimax with rotation. Discriminate validity is satisfactory when 
items show low loading on other constructs. Each component showed an eigenvalue 
greater than 1.0, the standard threshold for factor analysis. Ideally, all the components 


would load on separate factors and achieve an eigenvalue greater than 1.0. This did not 


44 Wikipedia page on Cronbach’s Alpha, http://en.wikipedia.org/wiki/Cronbach's_alpha, accessed 
June 18, 2007. 
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occur. However, the purpose of this research was to better understand the level of 
variance explained by variables impacting the Technological, Social, and Organizational 
constructs within the context of the TAM and as applied to the Asymmetrical Software 
Kit, as opposed to improving the model. When examining the variance explained by 
factors influencing PU and PEOU, both a sample size of 25 and higher than expected 
correlations between independent variables may be affecting the level of variance 


explained. 


Linear regression in SPSS was used to examine the relationships within the 
Technological, Organizational, and Social constructs of the TAM The hypothesized 
relationships delineated in this model were shown, by examining Beta Coefficient and 


Significance, to explain a significant portion of the variance in all three constructs. 


C: RESPONDENT DEMOGRAPHICS 


The overwhelming majority of our respondents (97%) were males, over the age of 
30 and with 12 or more years of service in the Army. Most of these were senior Non- 
Commissioned Officers (NCO’s) and Warrant Officers. They were split evenly between 
Military Intelligence (MI) and Special Forces Military Occupational Skills (MOS’s). On 
a whole, they were very well educated, with 82.4% having some college, and 34.2% 
having a College or Postgraduate degree. They are familiar with computers and use them 
regularly: 94.4% used their personal computer at least once a day, and 97.3% used their 
work computers once a day or more. A full 65% of the respondents indicated they had a 
year or more of experience using the ASK. For system training, 39.4% received their 
training on the ASK from the 18F/ Special Forces Intelligence Sergeant Course (SFISC), 
a 12 week course at Ft. Bragg, NC. Of the remaining respondents, 7.9% indicated they 
had received the two-week course from the New Equipment Training Team. Fully 52.7% 
of the respondents had received either no formal training, or only received On the Job 


Training (OJT) from their fellow analysts or supervisors. 


Most of the respondents interact with computers at work and at home on an 


almost daily basis. Their mean score for usage at home and work were 4.4 and 5.5 
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respectively. This indicates they use computers at home at least once a day, and use 
computers at work more than three to five times a day. Respondents were also asked to 
rank their usage of the components in the ASK (Orion Magic, Analyst Notebook, and 
ArcGIS). The table below displays their responses. A preponderance of the respondents 
used the software less than once a day. For Orion Magic and ArcGIS respectively, 64.8% 
and 52.6% rarely or never used the programs. The mean scores for component usage 
were low. ArcGIS and Analysts scored a mean reported usage of 2.8 each, which equates 
weekly usage. Orion Magic had a mean reported usage of 2.4, which is closer to monthly 


usage. 





10. Gurrent ASK Component Use 


About 
About About Thrseto §8=©More than 


once or Ries pone 
Nevar CaNce a once a five times five times P 
twlcea Count 
month Weak a day a day 
day 
oF .A% a? .O% 
Orion Magic 2 4358/5) IDe%id) 2781) 34% (3) aT 
(14) (70) 
20.0% 26.3% 
ANAS 1S (4) 21.1% (6) FB (a) 7.3% (3) 36 
(10) (10) ‘ ‘ 
; oe 7 27.0% aoa awe os 
Analyst's Notebook 21.6% (6) 24.3%/(9) 16.2% (6} 10) 54% (2) 5.4% (2) a7 
answered quesion 38 
Shipped guesion 0 
Figure 13. ASK Component Usage 


D. QUALITATIVE FEEDBACK: 


The survey contained three questions at the end requesting qualitative feedback 
on the ASK. The following are a sample of their responses to these questions. A 


complete listing can be found in Appendix C. 
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1. “What features of the ASK do you like most? Why?” 

- ArcGIS - industry leader to allow for utilization of GIS information. We need 
software that allows us to conduct analysis of a digital COP. ANB - very versatile 
application for link analysis. GeoRover - Makes utilizing ArcGIS a WHOLE LOT easier 


and much more efficient. 


- Orion Magic is a great help to researching local databases. ARCGIS has 
practical application in my job. Analyst Notebook helps to visualize intelligence 


reporting. 


- Orion Magic is one of the best search tools I have worked with. The problem is 


access to all the databases 


- | like ArcGIS for producing integrating data, performing analysis and producing 
shape files and databases; however, simpler more intuitive tools may be more useful for 
most common tasks related to spatial visualization (IE: Google Earth, etc). Allowing both 


spatial capabilities may bridge the gap between analysts and operators. 


Zi “What features of the ASK do you like the least? Why?” 
- ArcGIS is not easy to use, and requires extensive training. ArcGIS only works 
with powerful computers which combat units do not have. Finally, no help is provided to 


the users. 


- I detest Analyst Notebook because link charts become too crowded; charts 
cannot be easily searched for key words; and system does not allow for true social 
network analysis functions. I would recommend exploring and developing alternative 


link-analysis platforms. 


- ARCmap has too many features and too many different ways to do the same 
thing. This makes the program bigger and harder to use. When my PC locks up I can 


loose hours of work. 
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3. “What suggestions do you have for future improvements to the ASK 
concept?” 


- We need better computers, better training, and Imbedded support before and 


during deployments. 


- Bn S3's, 82's, and Officers who will be commanders at BN and Group level need 
to be familiar with ASK so they can effectively work with Intel fusion. It is very 
disheartening to spend over 100 man hours doing a 130pg SODARS with photo's, link 
charts, and ARCmap products to have it locked in the safe by your $3 CW4 that you 
submit it too so it is never accessible/searchable/submitted to the portal, or useful to 


anyone. 


- The concept of mapping people, places things and events across time and space 
is solid. However, the process requires re-design to ensure accessibility and acceptance. 
Recommend redesigning simple and intuitive data-centric systems based on technologies 


which present low barriers to entry and learning. 


E. RESULTS OF QUANTITATIVE ANALYSIS 


1. Previous Research 


When Hu, Lin, and Chen ran their study on software acceptance in a Law 
Enforcement context, their findings suggested a “prominent core influence path from 
efficiency gain to perceived usefulness and then to intention to accept.’’*> Lin also noted 
that “Perceived usefulness appears to be the single most acceptance driver for law 
enforcement officers, showing a highly prominent and significant direct effect on 
behavioral intention.”4© They also found the insignificance of perceived ease of use on 


perceived usefulness indicated a professional orientation amongst officers that had been 


4) Paul Hu, Cyienting Lin, & H. Chen, User acceptance of Intelligence and Security Informatics 
technology: A study of COPLINK, Journal of the American Society for Information Science and 
Technology, 2005; 56(3) p. 242. 


46 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, University of Arizona, 2004, p. 88. 
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suggested by earlier research.*’ They also found subjective norm had a significant effect 
on acceptance decision-making, but that normative beliefs themselves did not lead 


directly to user acceptance.*® 
p Results of this Study 


In this study, Perceived Usefulness exists, along with Perceived Ease of Use in 
the Technological context. The mean value of Perceived Usefulness was 5.550, meaning 
users perceived the ASK to be somewhat useful software. The mean of Job Relevance 
was 5.140 and Product Quality was 5.190. Efficiency Gain (EG) was 4.933, indicating 
the respondents may perceive a net loss of efficiency if they have to spend an inordinate 
amount of time manipulating an overly cumbersome system. This theme was carried 
over in the qualitative responses and during the interviews at Ft. Bragg. Using linear 
regression, a coefficient of determination (R’) and a regression coefficient (Beta) were 
determined for the model. For Perceived Usefulness, the adjusted R’ value was .716. Job 
Relevance and Product Quality were the most important factors. Efficiency Gain only 
had a small effect on Perceived Usefulness, and External Connectivity had little to no 


bearing on Perceived Usefulness. 


These results show that Hypotheses Two (The level of job relevance of the ASK 
as perceived by the tactical users will positively affect their perceived usefulness of the 
ASK) and Three (The level of product quality of the ASK as perceived by the tactical 
users will positively affect their perceived usefulness of the ASK.) were shown to be 
valid. Hypothesis One: (The level of efficiency gain of the ASK as perceived by the 
tactical users will positively affect their perceived usefulness of the ASK.) and 
Hypothesis 4: The level of external data connectivity of the ASK as perceived by the 


tactical users will positively affect their perceived usefulness of the ASK.) were found 


47 M.S. Larson, The Rise of Professionalism: A Sociological Analysis, Berkley, CA, University of 
California Press, 1977. 


48 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, University of Arizona, 2004, p. 89. 


34 


not to correlate as strongly. Statistically, Job Relevance and Product Quality are the two 


strongest determinants of Perceived Usefulness. 


Perceived Utility (PU) 


Efficiency Gain (EG) 


Job Relevance (JOBR) 


Product Quality PRODQ- | 5.720 1.137 
(PRODQ) PRODQ-2 _| 5.240 1.588 
PRODQ-3 4.400 Resp) 
PRODQ-4 5.400 L527 





Table 2... Reliability Analysis for Perceived Usefulness 


The mean score for Perceived Ease of Use was 4.690. The system is not 
perceived to be more than moderately easy to use. For Functionality and Learn ability, 
the mean scores 4.246 and 4.680 respectively. This indicates users found the ASK 
moderately functional in its layout, and rated their prospect of learning how to use the 
system at somewhat better than average. Of the factors bearing on Perceived Ease of 
Use, Functionality and Learn ability were the most important determinants of Perceived 
Ease of Use, Functionality correlated to Perceived Ease of Use at .841 and Learn ability 


correlated at .680. 


Hypotheses Six (The level of functionality of the ASK as perceived by the tactical 
users will positively affect their perceived ease of use of the ASK.) and Seven (The level 
of learn ability of the ASK as perceived by the tactical users will positively affect their 
perceived ease of use of the ASK.) were proven to have a statistically significant 
correlation. They are the strongest determinants of Perceived Ease of Use. Hypothesis 
Five (The level of external data connectivity of the ASK as perceived by the tactical users 


will positively affect their perceived ease of use of the ASK.) was again proven to be 
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uncorrelated to both factors in the Technological Construct. External Connectivity, while 
important to users, has no bearing on either Perceived Usefulness or Perceived Ease of 


Use. 


PEOU-1 4.080 1.552 
Perceived Ease of Use PEOU-2 4.880 1.394 0.875 
(PEOU) PEOU-3 4.800 1.471 ; 


PEOU-4 5.000 1.384 


FUNC-1 4.480 1.636 
; FUNC-2 4.320 1.700 
Functionality (FUNC) FUNC3 1.360 1680 0.905 


FUNC-4 3.880 1.763 


LEARN-1 4.760 1.640 
oe LEARN-2 4.720 1.882 
Learn ability (LEARN) LEARN 4.800 1390 0.739 


LEARN-4 4.440 1.709 


EXCON-1 5.320 2.201 
External Connectivity EXCON-2 5.680 1.651 0.759 
(EXCON) EXCON-3 5.720 2.051 


EXCON-4 5.360 





Table 3. = Reliability Analysis for Perceived Ease of Use 


(0.769) 
Adj R2 
 _a 


0.841 
0.695 
Ca (0.680) 


Figure 14. Technological Construct Regression Values 








(##)= Beta Coefficient 
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The mean of Subjective Norm was 4.400, indicating an overall moderate to weak 
influence of peers and leaders in influencing adoption behavior. Peer Influence mean 
was 4.733, and Chain of Command influence mean was 5.040. Users are influenced by 
their peers to a moderate extent, but not by their Chain of Command. Peer Influence 
correlates to Subjective Norm at .592, but Chain of Command influence correlates at - 
004. Tactical level SOF users are trained to work in small groups and rely on each for 
their survival. It is not surprising that soldiers trained to this extent would hold the 
opinions of their peers in higher regard than the opinions of their supervisors. It is very 
likely that command influence in a SOF unit is not as important a driver in a SOF unit as 


it would be in a conventional Army unit, but this bears further research. 


Hypothesis Eight (The level of peer influence (PEER) as perceived by the tactical 
users will positively affect the intensity of their perceived subjective norm towards 
accepting the ASK.) proved a statistically significant correlation. | Conversely, 
Hypothesis Nine (The level of command (COC) involvement as perceived by the tactical 
users will positively affect the intensity of their perceived subjective norm towards 
accepting the ASK.) correlated negatively. This may, as mentioned earlier, be a function 
of the way in which Special Operations Soldiers are trained to operate in small, self 
sufficient units far from the direct command and control exercised by many conventional 


military commands. 


2 
Se eon ie? 


Chain of Command COC-1 5.320 1.405 0.741 
(COC) COC-2 5.360 1.150 
COC-3 5.400 1.554 


COC-4 5.280 1.458 
PEER-2 4.920 1.605 
PEER-3 3.880 1.877 





Table 4. Reliability Analysis for Subjective Norm 


oi 





(0.592) 
Adj R2 
0.322 





(##)= Beta Coefficient 


Figure 15. Social Construct Regression Values 


In the Organizational construct, respondents displayed a Behavioral Intent to use 
mean of 5.346. This indicates a slightly positive intent to use the system. Support had a 
mean score of 4.040, indicating a neutral opinion of the support mechanisms for the 
software. Training also scored a mean of 4.550, which is neutral. However, when you 
reanalyze the data, SOF MOS soldiers scored a mean of 4.43 for Training, while MI 
MOS soldiers scored a mean of 4.64. This demonstrates a slight disparity in the 
perception of the training soldiers receive. System Availability scored in the negative 
region, with a mean of 3.160. Soldiers clearly thought system availability and finding 
enough computers appropriate to run the software were problem facing their acceptance 
of the system. These concerns were born out in the qualitative feedback from the survey. 
For Behavioral Intent, Training and Support were the most important factors. Training 
correlated to Behavioral Intent at .498, and Support correlated at .310. The adjusted R- 


squared value for the regression analysis of the Organizational construct was .215. These 
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low scores predict a likelihood of marginal ASK use by the respondents. This is born out 


in the usage data and the comments from the qualitative data. 


Hypothesis Ten (The level of system availability as perceived by the tactical users 
will positively affect the intensity of their behavioral intention towards accepting the 
ASK.) did not correlate strongly, although respondents did express much concern over 
the systems. Hypotheses Eleven (The quality of training as perceived by the tactical 
users will positively affect the intensity of their behavioral intention towards accepting 
the ASK.) and Twelve (The level of support as perceived by the tactical users will 
positively affect the intensity of their behavioral intention towards accepting the ASK.) 
proved the strongest correlations with Behavioral Intent. Training and Support, in that 


order, were the most important determinants. 


2.800 1.527 


3.360 1.468 


Support (SPT) 4.040 2.051 0.864 
4.120 1.943 
3.960 1.881 


Training (TNG) TNG-1 4.480 1.837 0.753 
TNG-2 4.280 1.744 
TNG-3 5.240 1.200 


TNG-4 4.200 





Table 5. — Reliability Analysis for Behavioral Intent 
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(0.310) Adj R 
0.215 


(0.498) 








(##)= Beta Coefficient 


Figure 16. Organizational Construct Regression Values 


F CONCLUSIONS AND DISCUSSION 


Findings from this analysis have several implications for research and technology 
fielding practices in the military context. The analysis suggests that Perceived 
Usefulness appears to be the most important acceptance driver for users at the Tactical 
level. Job Relevance and Product Quality were the two most significant drivers of 
Perceived Usefulness, suggesting that users may anchor their acceptance decision from a 
utility perspective. This was very similar to findings with law-enforcement officers and 


the COPLINK software.*? 


Second, Peer Influence seem to be the strongest driver of Subjective Norm. 
When assessing the utility of the ASK, Soldiers are likely to take into account the 


opinions of many people. What is significant here is that the Chain of Command held 


49 Chienting Lin, Examining Technology Usability and Acceptance in Digital Government: A Case 
Study in Law Enforcement, University of Arizona, 2004, p. 88. 
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almost no sway in a soldier’s decision. This may be due to the command’s lack of 


involvement with, or knowledge about the ASK. 


Finally, while System Availability is perceived as a problem among _ the 
respondents, it is not a significant driver of Behavioral Intent to use. Respondents had a 
lot to say about the availability and appropriateness of their computers in both pre-survey 
interviews at Ft. Bragg and in the qualitative portion of the survey itself. Training and 
technical Support for the ASK are, however, significant aspects. As noted before, there is 
a difference in perceptions of training amongst the respondents, and it’s split along MOS 
lines. Respondents also mentioned a perceived lack of dedicated tech support for the 


system, which hampered their efforts at proficiency during pre-mission train up cycles. 


Design and use of collaborative Intelligence analysis software like the ASK can 
greatly enhance the war fighting capabilities of SOF units. Any unit adopting new 
technology must carefully manage that process to reap the full benefit of the innovation. 
Driven by a need to explore technology acceptance by Tactical level users, this study 
designed and empirically tested a model for explaining individual users acceptance of the 
ASK. Overall, the model showed a good fit with the survey data collected, and it 
exhibited satisfactory explanatory power to most of the acceptance decisions observed. 
These findings will help instructors, software designers, and leaders design systems and 


fielding plans that foster, rather than stifle technology acceptance by Tactical users. 


4] 
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Vv. RECOMMENDATIONS AND CONCLUSIONS 


A. OVERVIEW 


From the beginning, the purpose of this research has been to identify those factors 
relating to user acceptance of the Asymmetrical Software Kit (ASK). This chapter will 
summarize the research, examine the implications of significant findings, and identify 
Opportunities for continued research in this field. The Technology Acceptance Model 
(TAM) results for this study are summarized, and some implications for further TAM 
usage are delineated. Finally, based off all the analysis, recommendations are given for 
continued training, fielding, and evolution of the ASK and the systems that will succeed 
it. 

With a better understanding of the key drivers of user acceptance for the tactical- 
level user, leaders, trainers, and procurement/futures professionals can quickly identify 
areas where user acceptance is currently being hindered, and likely to be hindered in the 
future. They can then plan acquisition, information, and training programs to erode these 
barriers and ensure a swift, smooth fielding and adoption. User training should 
concentrate on increasing Perceived Utility, as this is demonstrably the most important 
acceptance driver for ASK users. A utility centric view of technology is consistent with 
the findings of other research, and explains the insignificant effect of ease of use on 
Behavioral intent°°. It follows that users would not find an innovation useful simply 
because it 1s easy to use. Cultivating both peer and Chain of Command support for the 
new technology can create a normative reinforcement that encourages user acceptance as 
well. While this may not lead directly to acceptance, it can influence an individual 


analyst’s assessment of an innovation’s usefulness. 


50 Chienting Lin, Paul Hu, H. Chen, & J. Schroeder, Technology Implementation Management in Law 
Enforcement: COPLINK System Usability and User Acceptance Evaluations, Social Science Computer 
Review, Vol. 22, No. 1, 2004, p. 27.: 
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B RECOMMENDATIONS 


The intention of this research has been to use the findings to aid in the 
modification and management of the current Asymmetrical Software Kit, as well as the 


design and implementation of future software packages of this nature. 
1 System Training (TNG): 


Training, according to the results of the survey, is the number one discriminator 
for intent to use (BI). Military Intelligence (MI) soldiers who received their training 
through the Mobile Training Team or On the Job Training displayed a lower intent to use, 
a lower PEOU, a lower PU, and had a much lower perception of the value of the training 
they received on the system. This stands in stark contrast to the Special Forces Soldiers 
(MOS 18F and 180A), most of whom received their training through the Special Forces 
Intelligence Sergeants Course (SFISC). Their responses were the antithesis of the 
responses from MI soldiers. While both of them must use the system to communicate 
and share intelligence at the Tactical level, they do not receive the same training. The 
SFISC is a 12 week course that uses the ASK the entire time. The POI focuses on using 
the system in all aspects of the 18F’s job, and leads to a capstone project.°! The capstone 
is a culmination of their instruction on the ASK. Teams analyze immense amounts of 
real-world intelligence data and draw conclusions about active military conflicts. Some 
of these projects have discovered groundbreaking information and assisted the GWOT 


greatly. 


The disparity between the training the 18F’s receive in the SFISC and the training 
the MI soldiers receive is alarming, and may be the single largest driver in the disparity 
between their perceptions of the system and the adoption rates of the ASK. The disparity 
doesn’t have to exist, and much of that could be solved through similar, consistent, and 
cooperative training on the system. The current SFISC proposal to fix this problem 1s to 


have contractors teach their POI to MI soldiers and replicate the 18F course at another 


51 SFISC 18F Program of Instruction (POI) given to the author during interviews at Ft. Bragg, NC. 
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location. On the MI side of the house, the USASOC G72 has a project in concept stages 
called the ARSOF Military Intelligence Assignment Oriented Training (AOT) Initiative. 
This AOT is designed to provide incoming MI soldiers an in-depth training package to 


prepare them for the unique demands of the Army SOF environment.°? 


The recommendation of this author 1s that the proposals be combined. The SFISC 
would change to the Special Forces Intelligence Course, or SFIC. This training would 
include all MI MOS’s with the 18F’s to standardize unify the training they receive on the 
ASK. It is, after all, the system designed to help them analyze and communicate Tactical 
intelligence. USASOC trains communications equipment in a collaborative environment. 
Signal soldiers sit in the same classroom as the Special Forces Communication Sergeants 
(18E’s) and work together to establish communications. I would only be natural that the 
Special Forces Intelligence Sergeants should attend the same training as the Military 
Intelligence Soldiers that support them. The POI for the MI Soldiers could cover the new 
and unique expectations they will face, and the courses on the ASK could be a combined 
effort. MI soldiers and the 18F’s could collaborate on the capstone project, and use the 
ASK to piece together real world intelligence from ongoing conflicts and produce 
realistic products as they would do in combat. This would serve many ends. They MI 
and Special Forces MOS Soldiers would train together on the system they use on a daily 
basis to build their confidence in the system. Their collaboration would also boost their 
confidence in each others abilities, and promote trust in the soldiers they will have to 
work with on a daily basis. The relationships forged in these joint training environments 


are proven to be very strong bonds. 
2 Command Influence — Subjective Norm (SN): 


In “Best Practices in Counterinsurgency,” Kalev Sepp wrote that, “Intelligence 
operations that help detect terrorist insurgents for arrest and prosecution are the single 


most important practice to protect a population from threats to its security.”°3 This is a 


52 Emails and conversations with LTC Battle, USASOC deputy G-2 and SGM Coffman, USASOC 
G2 SGM about the AOT initiative. 


53 Kalev Sepp, Best Practices in Counterinsurgency, Military Review, May-June 2005, pp. 8-12. 
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change in the norm for many classically trained Intelligence professionals. According to 
Les Grau, a respected Counter-Insurgency theorist, Intelligence necessary for fighting 
insurgents is a radical shift, and a lot of change to swallow. “The military intelligence 
effort devoted to combating...insurgency has little in common with conventional 
intelligence operations in support of conventional maneuver war. Intelligence preparation 
of the battlefield, order of battle, templating forces, signals intelligence; measurement and 
signature intelligence; and electronic intelligence take different forms or are not 
applicable. The S2 or G2 has a different type of war and needs to take a different 
approach to dealing with it, much as the U.S. approach to peacekeeping evolved during 
the past decade.’>* Any change this large demands command involvement at a level 
even more than would be normally necessary to simply shepherd an innovation. This 
change isn’t just about a new system, it’s about a whole new way of analyzing 


Intelligence. 


When infusing a new innovation into an organization, military units need to plan 
and execute software fielding like a corporation. The current fielding model often ends at 
the purchase of software, neglecting several key steps to gain user buy-in. According to 
John P. Kotter, organizational transformations, including adoption of new software, can 
be fraught with peril. He studied eight factors that controlled success or failure of 
adaptation: Not enough sense of urgency, Not Creating a Powerful Enough Coalition, 
Lacking a Vision, Under communicating the Vision by a Factor of Ten, Not Removing 
Obstacles to the Vision, Not Systematically Planning for and Creating Short-Term Wins, 
Declaring Victory Too Soon, and Not Anchoring Changes in the Organization’s 
Culture.ss This data has shown that Chain of Command as having no correlation 
whatsoever to users acceptance decisions. It is obvious in this instance that the Chains of 
Command have under-communicated their vision, declared victory after fielding, and not 


institutionalized (anchored) this innovation in their organizational culture. 


54 Lester Grau, Something Old, Something New: Guerrillas, Terrorists, and Intelligence Analysis, 
Military Review, July-August 2004, accessed on 10 December 2007 from Small Wars Journal website at 
http://www.smallwarsjournal.com/documents/grau.pdf. 


55 John P. Kotter, “Leading Change: Why Transformation Efforts Fail,” Harvard Business Review, 
March-April 1995, pp. 59-67. 


46 


The results of measuring the Subjective Norm (SN) in this study show that 
leadership in USASOC is not a strong influence in Soldiers decisions to adopt the ASK. 
Peers, immediate supervisors, and the entire Chain of Command should be very 
important social influences on an operator, but they are not involved enough in this 
transformation of Intelligence. Educating the leadership about the benefits of the ASK, 
and helping them understand the importance of using the system is paramount. The 
Chain of Command must also mandate usage across the board, and remove old systems 
to assist in adoption. Hartwick and Barki found that mandatory usage was significantly 
related to Intention to Use. Mandatory users give greater weight to the opinions of 
others, using the system more frequently when their leadership thinks that usage is 
appropriate.-© In a military environment, a change like this should be simple enough. 
The chain of command can mandate usage of the ASK and have the legacy systems 
removed from the computers. In fact, many of the survey respondents and soldiers 
interviewed at Ft. Bragg admitted they still used Falconview, an older and simpler GIS, 
because their peers still used it and the Chain of Command tacitly supported this choice. 
Wilson stated that “Tasks that are familiar, easy, professionally rewarded, or well adapted 
to the circumstances in which the operators find themselves will be preferred because 
performing them is less costly than undertaking tasks that are new, difficult, or 
professionally unrewarded or that place the operator in conflict with his or her own 
environment.”°’ By demanding the Soldiers produce their Intelligence products with the 
ASK and removing competing systems like Falconview, commanders can greatly affect 
adoption rates in their units. These changes have to be swift and visible to all users, lest 


users continue to ignore the innovations around them. 


One of the suggestions brought out in the interviews was for the commanders to 
institute a weekly briefing. It was suggested that the program be called “Fox Friday” (for 
the 18 F Intel Sergeant MOS), and that the 18F’s and the MI Soldiers of the S2 (Intel) 


56 J. Hartwick and H. Barki, Examining the Role of User Participation in Information System Use, 
Management Science, Vol 40, (4), April 1994, pp. 440-465. 


57 J. Wilson, Bureaucracy: What Government Agencies Do and Why They Do It, New York, Basic 
Books, 1989, p. 231. 
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collaborate on a weekly briefing to the entire Battalion Chain of Command. A routine 
like this will do several things. First, it will demonstrate command knowledge of the 
system, and expectations of system use. Second, a program like this will go a long ways 
towards positively anchoring system usage in the culture of the unit and ensuring that 
new users are brought into the fold as they enter the unit. Having the SOF and MI MOS 
soldiers collaborate in a weekly preparation and briefing will cause them to work together 
towards a common goal, and strengthen trust and working relationships to stand the 


stresses of combat. 


3 System Availability (AV): 


The mean score for questions of system availability belied an overall negative 
impression of the number and type of systems available to run the software. Many of the 
respondents to the survey noted that there were concerns with both system availability, 
and system suitability. Soldiers expressed concerns that the number of systems their 
units were issued were insufficient to cover each and every user, and that many of the 
systems the ASK was loaded on were not fast enough to run complex, memory intensive 


programs like ArcGIS. 


During interviews at Ft. Bragg, one Warrant Officer admitted that he had upgrade 
the laptop by purchasing enough RAM and a fast enough processor for his 18F to use the 
system. His detachment CF-50 (Panasonic Toughbook Laptop) was two years old, had 
survived three rotations to Southern Afghanistan and was no longer performing at the 
level necessary to run memory intensive programs like ArcGIS without crashing. 
Another senior NCO noted that, during the SFISC, the computers they use are state of the 
art desktops maintained in a lab environment. They are hard-wired together and pull data 
and imagery off a dedicated server. Once he returned to his unit, his ASK laptop could 
not match the performance of the computers in the lab, and that he was greatly frustrated 
by this. One respondent even suggested using a COTS gaming computer like an 


Alienware™, with an extremely fast processor and extra RAM to run ArcGIS. 
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Issues like these have serious impacts on user acceptance, and can be easily 
remedied by updating Basis of Issue to reflect real needs, proper acquisition (with end 


user input) and property accountability. 
C. CONCURRENT RESEARCH: 


This project is only the first in a series being executed by the Naval Postgraduate 
School’s Common Operational Research Environment (CORE) Lab to investigate further 
improvements that might be made to the ASK. MAJ Derek McClain will be continuing 
this research, and attempting to identify standardized data models and database formats 


that will facilitate communication and speed analysis across USASOC. 
D. SUGGESTIONS FOR FURTHER STUDY 


As the problems addressed in this research are corrected and the ASK continues 
to evolve, the program should be evaluated again to examine new and improved features. 
TAM is an extremely effective tool for this type of ongoing evaluation. Future 
evaluations and scores can be conducted, and system evolution can be contrasted against 
this research as a baseline. IT 1s ever changing and evolving, and any lessons learned 
from this analysis require that further evaluation take place as the ASK goes through 
subsequent updates and improvements. Also, a user’s beliefs and attitude towards an 
innovation are likely to change and evolve over time.°® Therefore, it will be important to 
measure both initial acceptance, as well as continued acceptance of the system. New 
information technologies are continually being introduced to the community that could 
benefit from acceptance theory testing. Outside the realm of evaluating a new bit of 
software for the command, the ASK concept presents a plethora of IT management issues 


that could be studied and researched. 


58 E. Karahanna, D. Straub, & N. Chervany, Information Technology Adoption Across Time: A 
Cross-Sectional Comparison of Pre-Adoption and Post-Adoption Beliefs, MZS Quarterly, 23(2) 1999, pp. 
183-213. 
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E. CONCLUSIONS 


Design and use of advanced analytic technologies is paramount to enhancing the 
capabilities of USASOC Intelligence assets in the GWOT. To ensure these technologies 
are adopted and utilized to the fullest extent, the adopting units must manage user 
acceptance in the design, testing, fielding, and adoption. Motivated by the lack of 
previous research in this area, this study developed and empirically tested a model for 
explaining tactical-level users’ acceptance of the Asymmetrical Software Kit. Overall, 
the model demonstrated a good fit with the data collected, and exhibited satisfactory 
explanatory power to the technology acceptance decisions observed among Tactical level 


users of the ASK. 


The ASK was designed to enhance the analytic capabilities of the Special 
Operations Soldiers at the Tactical level of the GWOT. In light of the continuing 
fighting, and new and developing fronts in this “long war,” effective adoption strategies 
for needed innovations are especially crucial. Complex innovations that are rapidly 
fielded during wartime need special attention paid to their implementation lest they wind 


up expensive trinkets locked in a safe and forgotten. 


Intelligence analysis is an extremely complex series of tasks that have to be 
continuously coordinated. Recent advances in software have produced several tools that 
have excellent potential for application to the analysis and production process. The 
programs in the ASK have many tools and functions that make analysis easier and more 
effective for Intelligence professionals. To make full use of this concept, we must train 
ASK users together, to a common standard of interoperability, give them computers that 
are fast enough, and educate the Chain of Command to make use of the ASK a priority 


for Intelligence analysis at the Tactical level. 
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APPENDIX B- SURVEY QUESTIONS 


1. Introduction 


Naval Postgraduate School 
Informed Consent Statement 


Introduction: You are invited to participate in a study entitled “Assessing the acceptance and functional value of the 
Asymmetric Software Kit (ASK) at the tactical level” being conducted by the Department of Defense Analysis. 


Procedures: Please be frank and honest about your experiences with the ASK. This survey should take about 30 minutes of 
your time. It is completely anonymous, and no identifying data will be stored with your response. 


Risks and Benefits: This project does not involve greater then minimal risk and involves no known reasonably foreseeable 
risks or hazards greater than those encountered in everyday life. Benefits of the survey include a better understanding by 
the researcher of opinions and attitudes within the tactical level SOF community regarding the Asymmetric Software Kit. 


Compensation: There will be no tangible compensation provided for taking this survey. The results of the survey will be 
available from the USASOC G2 and in thesis form from the Naval Postgraduate School. 


Confidentiality & Privacy Act: All records of this study will be kept private. No information will be publicly accessible which could 
identify participants. This survey will be identified only as a code number on all research forms/data bases. Any records of 
participation will be maintained by for three years, after which they will be destroyed. 


Voluntary Nature of the Study. Participation is strictly voluntary, if you agree to participate, you are free to withdraw at any 
time without prejudice. 


Points of Contact. If there are any questions or comments regarding this project upon the completion of participation, 
contact the Principal Investigator, Richard Bergin, 831-656-3112, rdbergin@nps.edu, or researcher MAJ Peter H. Hopewell, 
910-489-4517, phhopewe@nps.edu. Any other questions or concerns may be addressed to the IRB Chair, LT Brent Olde, 
656-3807, baolde@nps.edu. 


Statement of Consent. I have been provided with a full explanation of the purpose, procedures, and duration of participation 
in this research project. I understand how my identification will be safeguarded and agree to participate in this study. I 
understand that by agreeing to participate in this research I do not waive any of my legal rights. This survey is anonymous 
and does not require any identifying personal data. Clicking the NEXT button below is your consent. 


yom PT =Vealele|e-le)allom Native) miit-i4ie)n| 


1. Gender 


Male 


Female 


2.MOS 


18F 
180A 
35D 
96B 
96D 
98c 
98G 
3516 


[__Jother (please specify) 





oe, 


3. Rank 


C Junior (es C ec C )e7 Ces ( wo1 C )ew2 C Jews C )ewa C )o1 C )o2 ( )o3 ( )o4 
Enlisted 
E1-E4 


4. Age 


( )1s-22 (23-25 C 26-30 (31-35 (36-40 C o+ 


5. Time in Service 


C )o-2 years C )s-5 years ( )o-s years C )o-11 years (12-15 years ( )16-18 years (19+ years 


6. Education 


( Jeep (xs Diploma (_)some College ( associate ( )ps/Ba ( postgraduate 


Degree degree 


7. Time/ Experience using ASK 


(1-2 months ( )3-6 months (7-12 months ( )13-18 months ( )19-24 months (2+ years 


8. Type of ASK training you received 
(New Equipment Fielding/ (_)1sF SFISC (Jon the Job Training (OJT) (None 


Mobile Training Team 
( other (please specify) 
| 


9. Current general computer usage 


About once a About once a About ance or Three to five More than five 
manth week twice a day times a day times a day 


persona Oo oOo Oo oO oO °C 
work Oo Oo oOo 6 6 6 


Never 


10. Current ASK Component Use 


About once a About once a About once or Three to five More than five 
month week twice a day times a day times a day 


Orion Magic C) C) 
ArcGIS C) 


Analyst's Notebook 


Never 


3. Technology Acceptance Modeling Survey 
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11. This survey is conducted using a 7-point Likert scale to state how much you 
agree or disagree with a statement. Please read each question carefully (some 
are negatives) and reply accordingly. Please evaluate the following statement 
for practice: 


The summer humidity in Fayetteville, NC doesn't bother me at all. 


Neither 

Strongly : Slightly Slightly Strongly 

; Agree with ; Agree nor : Disagree ; 

Agree with Agree with Disagree Disagree 

the Disagree with the 
the the : with the with the 
statement with the statement 

statement statement statement statement 

statement 


The summer humidity C) C) C) C) C) ©) O 


in Fayetteville, NC 
doesn't bother me at 
all. 


Comment: 


12. Using the ASK would improve my job performance 


: Neither 
Strongly : Slightly Slightly Strongly 
Agree with : Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the aes the peagree with the cae with the 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK would O C) C) C) C) C) © 


improve my job 
performance 


Comments: 


13. I could easily use any of the functions in the ASK on my own 


Neither 
. Slightly Slightly Strongly 
; Agree with . Agree nor : Disagree : 
Agree with tt Agree with Di Disagree ith th Disagree 
ye sagree Wi & 
the the : 9 with the with the 
statement with the statement 
statement statement statement statement 
statement 


I could easily use any C) C) C) C) C) OC) O 


of the functions in the 
ASK on my own 


Strongly 


Comment: 





oy 


14. It is frustrating to use the ASK because the computer I have access to is 


NOT fast enough 


Strongly 
Agree with 
the 
statement 


It is frustrating to use 
the ASK because the 
computer I have access 
to is NOT fast enough 


Comment: 


Agree with 
the 
statement 


Neither 
Agree nor 
Disagree 
with the 
statement 


Slightly 
Agree with 
the 
statement 


Slightly 
Disagree 
with the 
statement 


Disagree 
with the 
statement 


Strongly 

Disagree 

with the 
statement 


O OF OO O O O ODO O 


15. ASK’s interfaces make it easy for me to remember how to perform various 


kinds tasks 


Strongly 
Agree with 
the 
statement 


ASK’s interfaces make it 
easy for me to 
remember how to 
perform various kinds 
tasks 


Comment: 


Agree with 
the 
statement 


Neither 
Agree nor 
Disagree 
with the 
statement 


O OF O O O AO OO OC 


Slightly 
Agree with 
the 
statement 


Slightly 

Disagree 

with the 
statement 


Disagree 
with the 
statement 


Strongly 

Disagree 

with the 
statement 


16. Availability of computers for accessing the ASK is going to be a problem 


Strongly 
Agree with 
the 
statement 


Availability of computers 
for accessing the ASK is 
going to be a problem 


Comment: 


Agree with 
the 
statement 


Neither 
Slightly 


Agree with 
the 
statement 


Disagree 
with the 


statement 


J 


Agree nor 


Slightly 
Disagree 
with the 
statement 


17. My supervisor thinks I should NOT use the ASK 


Strongly 


Agree with 


the 


statement 


My supervisor thinks I 
should NOT use the ASK 


Comment: 


Agree with 
the 
statement 


Neith 
Slightly either 


Agree with 
the 
statement 


Disagree 
with the 


statement 


O OO OO OO O AO ODO O 
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Agree nor 


Slightly 

Disagree 

with the 
statement 


Disagree 
with the 


statement 


Disagree 
with the 


statement 


Strongly 

Disagree 

with the 
statement 


Strongly 

Disagree 
with the 
statement 











18. I do NOT feel comfortable using the ASK on my own 


Neither 
Strongly Slightly Slightly ; Strongly 
. ‘ Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the ene the ae aththe. wily ihe 
statement with the statement 
statement statement statement statement 
statement 


I do NOT feel C) C) C) C) C) C) C) 


comfortable using the 
ASK on my own 
Comment: 


Agree with 


19. Whenever possible, I would use the ASK for my job tasks 


; Neither ; 
Strongly Slightly Slightly Strongly 
; 3 Agree nor ; Disagree ; 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


Whenever possible, I C) OC C) OC) O O O 


would use the ASK for 
my job tasks 


Agree with 


Comment: 


| | 


20. There are no barriers to my using the ASK for my job tasks 


Neither 
Strongly : Slightly Slightly Strongly 
Agree with . Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the a the sek ia with the adie with the 
statement with the statement 
statement statement statement statement 
statement 


There are no barriers to C) C) C) C) C) C) C) 


my using the ASK for 
my job tasks 


Comment: 


21. ASK addresses my job related needs 


Neither 
Strongl Slight Slight! Strong| 
ahd Agree with . Agree nor hed Disagree ; ail 
Agree with Agree with Disagree Disagree 
the Disagree with the 
ne statement ue with the edi statement ite 
statement statement statement statement 
statement 


ASK addresses my job C) C) C) C) C) C) C) 


related needs 


Comment: 
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22. People who use the ASK do NOT tend to get recognized by their Chain of 
Command 


Neither 
Strongl Slightl Slight! Strong| 
atid Agree with . : Agree nor eed Disagree sia 
Agree with Agree with Disagree Disagree 
the Disagree with the 
ia statement He with the _ee statement wae 
statement statement statement statement 
statement 


People who use the ASK C) C) C) C) C) C) OC) C) 


do NOT tend to get 
recognized by their 
Chain of Command 


Comment: 


23. I would have no difficulty explaining why my use of the ASK may (or may 
not) improve my job performance 


; Neither ; 
: Slightly Slightly Strongly 

os Agree with : Agree nor ; Disagree ; 

Agree with . Agree with eid Disagree Disagree 
the Disagree with the 

the the . with the with the 
statement with the statement 
statement statement statement statement 
statement 


I would have no O O O 0 0 0 0 OO 


difficulty explaining why 
my use of the ASK may 
(or may not) improve 
my job performance 


Strongly 


Comment: 


24. ASK allows me to accomplish more work than would be otherwise possible 


Neither 
Strangl Slightl Slight Strong 
nid Agree with 9 y Agree nor siti Disagree stl 
Agree with Agree with Disagree Disagree 
the Disagree with the - 
the the 7 with the with the 
statement with the statement 
statement statement statement statement 
statement 


ASK allows me to C) C) C) C) C) ©) O © 


accomplish more work 
than would be otherwise 
possible 

Comment: 
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25. It would be undesirable for ASK to link to the relevant external databases 
(e.g. those in other intelligence agencies or other service intel components) 


Neither 
Strongl Slight Slight! Strongly 
wt Agree with 8 : Agree nor ated Disagree id 
Agree with Agree with Disagree Disagree 
the Disagree with the 
ia statement He with the _ee statement ilies 
statement statement statement statement 
statement 


It would be undesirable C) C) C) C) C) C) © 


for ASK to link to the 
relevant external 
databases (e.g. those 
in other intelligence 
agencies or other 
service intel 
components) 
Comment: 


26. Using ASK would be incompatible with the way I work 


Stronal Slight! Neither slight! Strongl 
ron 71] : I ron 
3 , Agree with 9 ¥ Agree nor alias Disagree ay 

Agree with Agree with Disagree Disagree 

the Disagree with the igs 
the the . with the with the 
statement with the statement 
statement statement statement statement 
statement 


ce en i 
way I work 


Comment: 


27. The various functions of the ASK programs (Orion Magic, Analyst's 
Notebook, and ArcGIS) are easily accessible 


Neither 
Strongly : Slightly Slightly ; Strongly 
; Agree with : Agree nor Disagree 
Agree with Agree with Disagree a Disagree 
et the th Disagree ah th with the et th 
e he with the with the 
statement with the statement 
statement statement statement statement 
statement 


The various functions of C) C) C) C) C) C) OC) 


the ASK programs 
(Orion Magic, Analyst's 
Notebook, and ArcGIS) 
are easily accessible 
Comment: 
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28. Using the ASK saves time 


Neither 
Strongly A Slightly Slightly ; Strongly 
Agree with Agree nor Disagree 
Agree with ‘ Agree with - Disagree Disagree 


the Disagree with the 
the the 7 with the with the 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK saves C) C) C) C) C) C) C) C) 


time 


Comment: 


29. My colleagues (fellow Analysts/18F’s) do NOT think I should use the ASK 


Strongl Slight! Neither slight Strongl 
_ Agree with : Agree nor ae Disagree ; kd 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the with the with the 


with th T 
statement acl U statement eee statement nearer statement 
statement 


My colleagues (fellow C) C) C) C) C) C) OC) C) 


Analysts/18F’s) do NOT 
think I should use the 
ASK 

Comment: 


30. I believe I could communicate to others the benefit of using the ASK 


: Neither - 
Strongly , Slightly Slightly ; Strongly 
; Agree with : Agree nor ; Disagree ; 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the ; with the with the 
statement with the statement 


statement statement statement statement 
statement 


I believe I could O OC) C) © ©) O O 


communicate to others 
the benefit of using the 
ASK 


Comment: 


a! 


31. ASK does NOT address the critical aspects of my job 


Neither 
Strongl Slightl Slight! Strongly 
Y Agree with ie J Agree nor om Disagree iil 
Agree with th Agree with Di : Disagree with th Disagree 
the "4 the oe with the ' with the 


statement with the statement 
statement statement statement statement 
statement 


ASK does NOT address = (_) C) C) C) C) C) © © 


the critical aspects of 
my job 


Comment: 


Pp 
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32. I would use ASK more if I knew my boss wanted me to 


I would use ASK more if 
I knew my boss wanted 
me to 


Comment: 


Neither 
Agree nor 
Disagree 
with the 
statement 


O OF OF OO O OO O 


Strongly 
Agree with 
the 
statement 


Slightly 
Agree with 
the 
statement 


Slightly 
Disagree 
with the 
statement 


Strongly 

Disagree 

with the 
statement 


Agree with 
the 
statement 


Disagree 
with the 
statement 


33. The computer I have is slow and makes using the ASK difficult 


The computer I have is 
slow and makes using 
the ASK difficult 


Comment: 


Neither 
Agree nor 


Slightly 


Agree with 
the Disagree 


with the 
statement 


so «ee fe 86 ® © 


Slightly 

Disagree 

with the 
statement 


Strongly 

Disagree 

with the 
statement 


Strongly 
Agree with 
the 
statement 


Agree with 


the 
statement 


statement 


34. When I have access to ASK, I would use it as often as needed 


When I have access to 
ASK, I would use it as 
often as needed 


Comment: 











Neither 
Agree nor 
Disagree 
with the 
statement 


Ce en rn 


Strongly 
Agree with 
the 
statement 


Slightly 
Agree with 
the 
statement 


Slightly 
Disagree 
with the 
statement 


Strongly 

Disagree 
with the 
statement 


Agree with 
the 
statement 


Disagree 
with the 
statement 


35. Accessing relevant Intelligence databases through the ASK is NOT 


important 


Accessing relevant 
Intelligence databases 
through the ASK is NOT 
important 


Comment: 


Slightly Neither 
Agree with Agree nor 
? Agree with . 


the Disagree 


the 
statement with the 
statement 
statement 


O OO OF O O OQ O 


Strongly 
Agree with 
the 
statement 


Slightly 
Disagree 
with the 
statement 


Strongly 

Disagree 

with the 
statement 


Disagree 
with the 
statement 
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36. Using ASK would NOT make it easier to do my job 


Neither 
Strongl Slightl Slightl Strongl 
ahd Agree with 8 : Agree nor a Disagree id 
Agree with . Agree with Disagree Disagree 
the hie the a he | ki ihe 
statement with the statement 
statement statement statement statement 
statement 


Using ASK would NOT C) C) C) C) C) OC O © 


make it easier to do my 
job 


Comment: 


37. I would be able to use the ASK effectively even if there is no one around to 
show me how to do it 


Strongl Slightl none sisi Strongl 
Ai ' 
A saeels Agree with A 9 1 Agree nor DI ae Disagree Di ae 
ree wit! ree wi isagree isagree 
4 the 4 Disagree 4 with the 4 


the the : with the with the 
statement with the statement 
statement statement statement statement 
statement 


I would be able to use C) C) C) ©) C) O O 


the ASK effectively even 
if there is no one 
around to show me how 
to do it 


Comment: 


38. In my job, using the ASK is important 


Neither 
Strongl Slightl Slight Strongl 
and Agree with “ Agree nor mid Disagree 9 J 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the 7 with the with the 
statement with the statement 
statement statement statement statement 
statement 


In my job, using the C) C) C) C) C) C) C) 


ASK is important 


Comment: 


po 


39. The quality of the products I am able produce with the ASK is high 


Neither 
| li | lig! ] 
laa Agree with = ightly Agree nor any Disagree sokalliald 
Agree with Agree with Disagree with the Disagree 


on the a BSGrEe eh the with the 


statement with the statement 
statement statement statement statement 
statement 


The quality of the C) C) C) C) C) C) C) 


products I am able 
produce with the ASK is 
high 

Comment: 
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40. I have difficulty finding a computer to use ASK when I need it 


Neither 
Strongl Slightl Slightl Strongl 
ahd Agree with 8 : Agree nor a Disagree id 
Agree with . Agree with Disagree Disagree 
the hie the a he | whe 
statement with the statement 
statement statement statement statement 
statement 


I have difficulty finding) C) C) C) C) © ©) 


a computer to use ASK 
when I need it 


Comment: 


41. It would be easy for me to become skillful in using the ASK 


Neither 
Strongl Slightl Slight Strong 

ay Agree with 9 y Agree nor aiid aly 

Agree with th Agree with Di Disagree Disagree 
ne Sagree 

the the ‘ 9 with the with the 
statement with the 

statement statement statement statement 
statement 


It would be easy for me C) C) C) C) C) C) © 


to become skillful in 
using the ASK 
Comment: 


42. It is easy for me to find the information and commands I need on the ASK 


screens 


Neither 
Stronal Slightl Slight Strong| 
oY Agree with 9 ¥ Agree nor ee Disagree cai 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the : with the with the 
statement with the statement 
statement statement statement statement 
statement 


It is easy formetofind (_) OC) C) C) C) e ® 


the information and 
commands I need on 
the ASK screens 


Comment: 


po 


43. Interacting with the ASK does NOT require a lot of mental effort 


Neither 
| li | lig! ' 
lacie Agree with = amntly Agree nor Aan Disagree sohalliald 
Agree with Agree with Disagree Disagree 
the ie the Disagrees ah the Ee the 
statement with the statement 
statement statement statement statement 
statement 


Interacting with the ASK C) C) C) C) C) C) C) 


does NOT require a lot 
of mental effort 


Comment: 
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44. The ASK constantly produces substandard Intel Products for my effort 


Neither 
Strongl Slightl Slightl Strongl 
atl Agree with 8 : Agree nor a Disagree ad 
Agree with . Agree with Disagree Disagree 
the hie the a he | wkiihe 
statement with the statement 
statement statement statement statement 
statement 


The ASK constantly C) C) C) C) C) C) © 


produces substandard 
Intel Products for my 
effort 


Comment: 


45. ASK is rigid and inflexible to interact with 


; Neither _ 
Strongly : Slightly Slightly ; Strongly 
Agree with eaire wkN Agree with oo Disagree eaetee Disagree 
a the a q with the a 


Disagree 
the the : 4 with the with the 
statement with the statement 
statement statement statement statement 
statement 


ASK is rigid and C) C) C) C) C) ©) O 


inflexible to interact with 
Comment: 


46. My interaction with the ASK is clear and understandable 


: Neither - 
Strongly , Slightly Slightly ; Strongly 
; Agree with : Agree nor ; Disagree ; 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


My interaction with the C) OC) C) C) C) © © 


ASK is clear and 
understandable 


Comment: 








47. I would be more inclined to use the ASK if my co workers (other Analysts or 
18F’s) were already using it 


Neither 
Strangl Slightl Slight Strong| 

Agree nn Mires eH A tes oa Agree nor ie = PIsagree Disa nil 
| 

: the ° Disagree 7 with the 

the the . with the with the 

statement with the statement 

statement statement statement statement 

statement 


I would be more C) C) C) C) C) © © © 


inclined to use the ASK 
if my co workers (other 
Analysts or 18F's) were 
already using it 
Comment: 
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48. Overall, it is a bad idea to use the ASK in my job 


Neither 
Strongl Slight Slight! Strong| 
atl Agree with 8 : Agree nor =? Disagree id 
Agree with Agree with Disagree Disagree 
the oo the in ae Te 
statement with the statement 
statement statement statement statement 
statement 


Overall, it is a bad idea C) C) C) C) C) C) O 


to use the ASK in my 
job 


Comment: 


49. People who influence my work think I should use the ASK 


Neither 
Strongl Slightl Slight Strong 

ay Agree with 9 y Agree nor aiid i aly 

Agree with th Agree with Di Disagree Disagree 
ne Sagree 

the the ‘ 9 with the with the 
statement with the 

statement statement statement statement 

statement 


People who influence C) C) (i) C) C) ©) © 


my work think I should 
use the ASK 


Comment: 


50. People who evaluate my job performance think I should NOT use the ASK 


; Neither - 

Strongly : Slightly Slightly ; Strongly 

; Agree with : Agree nor ; Disagree ; 

Agree with Agree with Disagree Disagree 
the Disagree with the 

the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


People who evaluatemy — (_) C) C) C) C) © © 


job performance think I 
should NOT use the ASK 


Comment: 








__| 


51. Overall, I find the ASK to be useful to my job 


Neither 
Strongl Slight! Slightl Strong 
si Agree with " Agree nor ee Disagree aid 
Agree with Agree with Disagree Disagree 
the Disagree with the 
he statement Hie with the aioli statement sie 
statement statement statement statement 
statement 


Overall, I find the ASK C) C) C) C) C) OC) © © 


to be useful to my job 
Comment: 
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52.1 find the user interfaces in the ASK easy to learn 


Neither 
Strongl Slight Slightl Strong| 
atl Agree with 8 : Agree nor a Disagree id 
Agree with Agree with Disagree Disagree 
the = the in ae Te ™ 
statement with the statement 
statement statement statement statement 
statement 


I find the user C) C) C) C) C) C) O 


interfaces in the ASK 
easy to learn 


Comment: 


53.In my job, use of the ASK is relevant 


Neither 
Strongly Slightly eae Slightly Strongly 
Agree with Agree with g | Disagree Disagree 
the Disagree ; 
‘ with the with the 
statement with the 


the the 
statement statement statement statement 
statement 


In my job, use of the C) C) C) C) C) C) © 


ASK is relevant 


Agree with 


Comment: 


54. Using the ASK would fit into my work style 


Neither 
St | Slight Slight St | 
reney Agree with ed Agree nor ee Disagree hidintil 
Agree with Disagree 


Agree with Disagree 
the Disagree with the 
the the i; with the with the 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK would fit (+) OC C) C) C) C) © ©) 


into my work style 


N/A 


Comment: 


| 


55. There are NOT enough computers/software/hardware for everyone to 
use ASK 


Neither 
Strongl Slight Slight! Strongl 
atl Agree with " Agree nor ee Disagree al 
Agree with Agree with Disagree Disagree 
the Disagree with the NSA 
the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


There are NOT enough C) C) C) C) C) C) C) © 


computers/software/hardware 
for everyone to use ASK 


Comment: 





68 


56. My immediate supervisor thinks I should NOT use the ASK 


Neither 
Strongl Slightl Slightl Strongl 
ahd Agree with 8 : Agree nor a Disagree id 
Agree with . Agree with Disagree Disagree 
the hie the a he | while 
statement with the statement 
statement statement statement statement 
statement 


My immediate C) C) C) C) C) C) O 


supervisor thinks I 
should NOT use the ASK 


Comment: 


57.In general, Iam NOT happy with the products produced with the ASK 


; Neither . 

Strongly Slightly ince nee Slightly Strongly 

Agree with th Agree with a | Disagree Disagree 
he Isagqree 

the the ‘ 9 with the with the 
statement with the 

statement statement statement statement 
statement 


In general, I am NOT C) C) (i) C) C) C) © 


happy with the products 
produced with the ASK 


Comment: 


Agree with 


58. The people I work with think that I should use the ASK 


: Neither - 
Strongly , Slightly Slightly ; Strongly 
; Agree with : Agree nor ; Disagree ; 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


The people I work with C) OC) C) C) C) C) © 


think that I should use 
the ASK 


Comment: 


| 


59. The benefits of using the ASK are NOT apparent to me 


Neither 
Stroangl Slightl Slightl Strong 
_ Agree with " Agree nor ama Disagree aid 
Agree with Agree with Disagree Disagree 
the Disagree with the 
he statement Hie with the aioli statement sie 
statement statement statement statement 
statement 


The benefits of using C) C) C) C) C) C) © 


the ASK are NOT 
apparent to me 


Comment: 
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60. Using the ASK would enhance my effectiveness at work 


Neither 
Strongl Slightl Slightl Strongl 
atl Agree with 8 Agree nor a Disagree id 
Agree with . Agree with Disagree Disagree 
the hie the coe he oo. | wna 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK would OC) C) C) C) C) © © 


enhance my 
effectiveness at work 


Comment: 


61. It is easy to navigate between the different search queries, output 
products, and function screens on the ASK 


Strongl Slight! Neher Stight Strong| 
Md \ 
denies Agree with g y Agree nor ; ay Disagree ; ae 
Agree with Agree with ; Disagree Disagree 
the Disagree with the 


the the : with the with the 
statement with the statement 
statement statement statement statement 
statement 


It is easy to navigate C) C) C) C) C) ©) © 


between the different 

search queries, output 
products, and function 
screens on the ASK 


Comment: 


62. The ASK is incompatible with the way I work 


Neither 
Strongl Slightl Slight Strongl 
and Agree with “ Agree nor amid Disagree 9 J 
Agree with . Agree with Disagree Disagree 
the Disagree with the 
the the 7 with the with the 
statement with the statement 
statement statement statement statement 
statement 


The ASK is incompatible (_) C) C) C) OC) © O 


with the way I work 


Comment: 


po 


63. The individuals with whom I work routinely think I should use the ASK 


Neither 
| li | li 
pee Agree with = ightly Agree nor ‘ ightly Disagree plata 
Agree with Agree with Disagree with the Disagree 


an the the Disagree with the with the 


statement with the statement 
statement statement statement statement 
statement 


The individuals with O C) C) C) C) C) C) 


whom I work routinely 
think I should use the 
ASK 


Comment: 
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64. The ASK runs slowly on the computer I have access to because the 


machine it too old/ worn out 


Strongly 
Agree with 
the 
statement 


Agree with 
the 
statement 


The ASK runs slowly on 
the computer I have 
access to because the 
machine it too old/ worn 
out 


Comment: 


Slightly 


Agree with 


the 


statement 


Neither 
Agree nor 
Disagree 
with the 


statement 


O OF OF OO O OO OO 


Slightly 
Disagree 
with the 


statement 


Disagree 
with the 


statement 


Strongly 
Disagree 
with the 


statement 


65. If I need someone's help using the ASK, I can NOT easily get it 


Strongly 
Agree with 
the 
statement 


Agree with 
the 
statement 


If I need someone's 
help using the ASK, I 
can NOT easily get it 


Comment: 


Slightly 


Agree with 


the 


statement 


Neither 
Agree nor 


Disagree 
with the 
statement 


Cr rn en) 


Slightly 
Disagree 
with the 


statement 


Disagree 
with the 


statement 


66. I am NOT able to use the ASK in my job when I want to 


Strongly 
Agree with 
the 
statement 


Agree with 
the 
statement 


Iam NOT able to use 
the ASK in my job when 
I want to 


Comment: 


67. The ASK training sessions and tutorials that I received are NOT helpful 


Strongly 
Agree with 
the 
statement 


Agree with 
the 
statement 


The ASK training 
sessions and tutorials 
that I received are NOT 
helpful 

Comment: 


Slightly 


Agree with 


the 


statement 


Slightly 
Agree with 
the 
statement 
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Neither 
Agree nor 
Disagree 
with the 


statement 


O O OF O O OO O 


Neither 
Agree nor 
Disagree 
with the 
statement 


Slightly 
Disagree 
with the 


statement 


Slightly 

Disagree 

with the 
statement 


Disagree 
with the 


statement 


Disagree 
with the 
statement 


Strongly 
Disagree 
with the 


statement 


O 


Strongly 
Disagree 
with the 


statement 


Strongly 

Disagree 

with the 
statement 


N/A 


a n\n a) 





68. I have received adequate training on the ASK 


Neither 
Stroangl Slightl Slightl Strong 
Agree on alle A a af irideciaaad es wa BIsagree Disa a 
| 
the . Disagree with the ais 
the the with the with the 


with th 
SraceMmieaie statement ee statement sacl statement 


statement 


I have received C) C) C) C) C) © ©) 


adequate training on 
the ASK 
Comment: 


statement 


69. Using the ASK would be unpleasant 


Neither 
Strongly : Slightly Slightly ; Strongly 
: Agree with : Agree nor ; Disagree ; 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the : with the with the 
statement with the statement 
statement statement statement 
statement 


Using the ASK would be (_) C) C) C) C) ©) ©) 


unpleasant 


statement 


Comment: 


, | 


70. Using the ASK in my job would NOT increase my productivity 


Neither 
Strongly ; Slightly Slightly Strongly 
“Agree with j : Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the - with the with the 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK in my job (_) C) C) OC) © O O 


would NOT increase my 
productivity 


Comment: 


as 


71. Learning to operate the ASK would be easy for me 


Neither 
Strangl Slightl Slight! Strongl 
= Agree with " . Agree nor on Disagree aid 
Agree with Agree with Disagree Disagree 
the whiz the eee aye. own the 
statement with the statement 


statement statement statement statement 
statement 


Learning to operate the C) C) C) C) C) C) C) 


ASK would be easy for 
me 


Comment: 
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72. I have no problem telling others about the satisfactory results of using the 
ASK 


Neither 
Strongl Slight! Slightl Strong 
atid Agree with . : Agree nor whi Disagree , sid 
Agree with Agree with Disagree Disagree 
the Disagree with the 
ane statement He with the =e statement wae 
statement statement statement statement 
statement 


I have no problem C) C) C) C) C) C) ©) 


telling others about the 
satisfactory results of 
using the ASK 
Comment: 


73.I have problems getting help when using the ASK 


. Neither - 
Strongly . Slightly Slightly Strongly 
Agree with ; Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the . with the with the 
statement with the statement 
statement statement statement statement 
statement 


I have problems getting C) C) C) C) C) ©) O 


help when using the 
ASK 


Comment: 


| 


74. The ASK allows me to accomplish more tasks quickly 


Neither 
Stronal Slightl Slightl Strongl 
and Agree with g ¥ Agree nor $ ’ Disagree wy 
Agree with Disagree Disagree 
the Disagree with the 
the - with the with the 
statement with the statement 
statement statement statement statement 
statement 


The ASK allows me to C) C) C) C) C) ©) © 


accomplish more tasks 
quickly 


Comment: 


po 


75. Using the ASK would NOT be beneficial to my work 


Neither 
| li | lig! ' 
eee Agree with = amntly Agree nor any Disagree okalliald 
Agree with Agree with Disagree Disagree 
the ie the Disagrees an the TEE se the 
statement with the statement 
statement statement statement statement 
statement 


Using the ASK would C) C) C) C) C) © © 


NOT be beneficial to my 
work 


Comment: 
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76. Linking the ASK to Databases resident in other Intel Agencies is desirable 


Neither 
Strongl Slightl Slightl Strongl 
atl Agree with 8 : Agree nor cna Disagree id 
Agree with . Agree with Disagree Disagree 
the hie the a he | whe 
statement with the statement 
statement statement statement statement 
statement 


Linking the ASK to C) C) C) C) C) C) © © 


Databases resident in 
other Intel Agencies is 
desirable 
Comment: 


77. Linking ASK to Databases resident in other USASOC units is NOT important 


; Neither 2 
Strongly : Slightly Slightly ; Strongly 
Agree with eaire WEN Agree with si sabanall cl Disagree eaetee Disagree 
a the a q with the a 


Disagree 
the the : 4 with the with the 
statement with the statement 
statement statement statement statement 
statement 


Linking ASK to C) C) C) C) C) C) O 


Databases resident in 
other USASOC units is 
NOT important 
Comment: 








| 


78. Overall, I find ASK easy to use 


Neither 
Strongly : Slightly Slightly Strongly 
Agree with : Agree nor Disagree ; 
Agree with Agree with Disagree Disagree 
the sae the peda with the iets with the 
statement with the statement 
statement statement statement statement 
statement 


Overall, I find ASKeasy  (_) C) C) C) C) e O © 


to use 


Comment: 


a! 


79. Given that I have access to ASK, I intend to use it to accomplish my duties 


as often as needed 


Neither 
| li | lig! ' 
lacie Agree with = antl Agree nor ia Disagree sokalliald 
Agree with Agree with Disagree Disagree 
the ie the Disagrees ah the Ee the 
statement with the statement 
statement statement statement statement 
statement 


Given that I have C) C) C) C) C) C) O © 


access to ASK, I intend 
to use it to accomplish 
my duties as often as 

needed 

Comment: 


po 
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80. I find it easy to get the ASK to do what I want it to do 


Neither 
Strongl Slightl Slight! Strongl 
wd Agree with aie Agree nor ceil Disagree ald 
Agree with Disagree 


Agree with Disagree 
et the a tk Disagree it - with the th t 
12 1 WIT ee WIth CMe 
mm with th 
statement een statement ene statement Soren statement 
statement 


I find it easy to get the C) C) C) C) C) OC O OC 


ASK to do what I want it 
to do 


Comment: 


81. There are NOT enough computers in my unit for me to use ASK when I 
need to 


Neither 
Strongly Slightly Slightly Strongly 
Agree with : Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the Disagree asi with the NSA 
the the . with the with the 
statement with the statement 
statement statement statement statement 
statement 


There are NOT enough C) C) C) ©) C) ©) OC ©) 


computers in my unit 
for me to use ASK when 
I need to 


Comment: 





82. Using the ASK reduces the time I spend completing my job-related tasks 


Neither 
Strongly - Slightly ig Strongly 
Agree with ; Agree nor Disagree 
Agree with Agree with Disagree 
the Disagree with the 
se statement alia with the = statement ee 
statement statement statement statement 
statement 


Using the ASK reduces C) C) C) C) C) C) OC) C) 


the time I spend 
completing my job- 
related tasks 
Comment: 
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83.Iam NOT pleased with the quality of the Intel products I can generate with 
the ASK 


Neither 
Strongl Slight Slight! Strong| 
= Agree with . : Agree nor ail Disagree , _ 
Agree with Agree with Disagree Disagree 
the Disagree with the 
she statement “ with the Boabhain statement noha 
statement statement statement statement 
statement 


1 am NOT pleased with C) C) C) C) C) C) C) C) 


the quality of the Intel 
products I can generate 
with the ASK 


Comment: 


84. I feel comfortable with the level of in-house support available for my using 
the ASK 


; Neither ; 
. Slightly Slightly Strongly 

; Agree with : Agree nor ; Disagree ; 

Agree with Agree with Disagree Disagree 
the Disagree with the 

the the i with the with the 
statement with the statement 
statement statement statement statement 
statement 


I feel comfortable with C) C) C) C) @ © © 


the level of in-house 
support available for my 
using the ASK 


Strongly 


Comment: 


85. Ido NOT have access to the resources I would need to use the ASK 
successfully in my job 


Neither 
rn | i li 
aahchiaes ¥ Agree with = ghtly Agree nor sanity Disagree mba 
Agree with Agree with Disagree = Disagree 
the les the isagree eh the ith the 
statement with the statement 
statement statement statement statement 
statement 


Ido NOT have access to C) C) C) C) C) C) OC) 


the resources I would 
need to use the ASK 
successfully in my job 
Comment: 





76 

















86. The computer I have access to has adequate processing power to run the 
ASK 


Neither 
Strangl Slight! Slight Strong 
ind Agree with : Agree nor sped Disagree nid 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the ee the en the with the seerentenk with the 
statement statement statement statement 
statement 


The computer I have C) C) C) C) C) C) © 


access to has adequate 
processing power to run 
the ASK 

Comment: 


87. To the extent possible, I intend to use the ASK in my job 


; Neither on 
Strongly . Slightly Slightly 
Agree with : Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the . with the with the 
statement with the statement 
statement statement statement statement 
statement 


To the extent possible, C) C) C) C) C) C) C) 


I intend to use the ASK 
in my job 


Strongly 


Comment: 


| | 


88. The organization of information on the ASK screens is clear and intuitive 


Neither 
Stronal Slight! Slight Strong| 
and Agree with ¥ ¥ Agree nor wehini Disagree wy 
Agree with . Agree with Disagree Disagree 
the Disagree with the 
the the ; with the with the 
statement with the statement 
statement statement statement statement 
statement 


The organization of C) C) C) C) C) C) © O 


information on the ASK 
screens is clear and 
intuitive 


Comment: 


89. I have the opportunities and knowledge I need to use the ASK in my job 


Strongl Slight! Neither slight! Strongl 
ron i i ron 
ae Agree with : : Agree nor iid Disagree ; ie 
Agree with Agree with ; Disagree Disagree 
the Disagree with the 
the the - with the with the 
statement with the statement 
statement statement statement statement 
statement 


I have the opportunities  (_) C) C) C) C) C) © © 


and knowledge I need 
to use the ASK in my 
job 


N/A 


Comment: 


Pp 





7T7/ 


90. The layout for the various ASK program screens (Orion Magic, Analyst’s 
Notebook, and ArcGIS) are NOT clear and consistent 


, Neither on 
Strongly ; Slightly Slightly ; Strongly 
Agree with pares with Agree with pares noe Disagree eee Disagree 
the ) fee with the 


Disagree 
the the . 9 with the with the 
statement with the statement 
statement statement statement statement 
statement 


The layout for the C) C) C) C) C) C) ©) C) 


various ASK program 
screens (Orion Magic, 
Analyst's Notebook, and 
ArcGIS) are NOT clear 
and consistent 


Comment: 


91. I am NOT getting the training I need to be able to use the ASK effectively 


Neither 
Strongly . Slightly Slightly Strongly 
Agree with ; Agree nor Disagree 
Agree with Agree with Disagree Disagree 
the Disagree with the 
the the - with the with the 
statement with the statement 
statement statement statement statement 
statement 


I am NOT getting the C) C) C) C) C) C) C) C) 


training I need to be 
able to use the ASK 
effectively 
Comment: 





92. What features of the ASK do you like the most? Why? 


=z 


93. What features of the ASK do you like the least? Why? 





wi 


94. What suggestions do you have for future improvements to the ASK 
concept? 


id | 


Thank You very much for your time. If you have any questions about the project, or would like a copy of my thesis when it is 
complete, please contact me at phhopewe@nps.edu. 
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